This repository contains code I wrote in the class Empirical Methods in Natural Language Processing.
Hopefully this can be useful to other people who are also learning NLP techniques. The code that I wrote is licensed under WTFPL version 2.
The details are described in the report.pdf
files in each sub-folder. Here's a brief summary of what they are about:
- hw1: Restore vowels to vowelless text using Finite-State Machines
- hw2: Trigram language model with improved Kneser-Ney smoothing (best performance in class)
- hw3: Unsupervised part-of-speech tagging using Baum–Welch Algorithm
- hw4: Part-of-speech tagging using Conditional Random Fields (3rd best performance in class)
- hw5: Context free grammar parsing using CKY algorithm, with vertical markovization
- final project: a modern-Chinese-to-classical-Chinese machine translation system
Most code is written in scala (except for hw5, which is written Python: the professor kindly provided a few hundred lines of useful code in Python and I was too lazy to reimplement them, so I just learnt the language instead). You may need sbt or sbt script runner to execute them.