Reversible tokenization in Python.
j
revtok
public
61 stars
5 forks
1 issues
Commits
List of commits on branch master.Verified
f1998b72a941d1e5f9578a66dc1c20b01913caabintern strings (#4)
jjekbradbury committed 6 years ago
Unverified
852fd955132248bbde7b18236db27a78feb314a2bump version
vvzhong committed 7 years ago
Verified
14418dae53197da608df342c11afb35fc6780938always separate punctuation as own token (#3)
jjekbradbury committed 7 years ago
Verified
f74500ee2e7bf2f309ea9200c2f0955971496240fix decap
jjekbradbury committed 7 years ago
Unverified
19ab3dee01fc6f49af87c2211f7e64cead155767bump version
jjekbradbury committed 7 years ago
Unverified
105750d71157a7e261762f193fe632da1565f0c5better decap
jjekbradbury committed 7 years ago