CRFSparse -- A Proximal Quasi-Newton Optimizer for L1-regularized CRFs.


Description

CRFSparse is a proximal Quasi-Newton solver for minimizing L1-regularized conditional negative log-likelihood of Conditional Random Fields(CRF), in particular, for Sequence Labeling and Hierarchical Classification problems.

This solver can also be used to optimize any differentiable convex loss functions with a L1-regularization, i.e.,
     f(w) = l(w) + \lambda |w|_1,
where l(w) is a convex differentiable loss function and |w|_1 is L1 norm of w. For users who want to implement their own models, please see the section, "Implementation for other problems", in the README file in the package.

Download and Compile

You can download the code here.

Compile it by "make" in a unix system with g++.


Usage

Training:
./train (options) [train_data] (model)
	
options:
	-p problem_type: (default 1)
		1 -- Sequence labeling (info file = feature template)
		2 -- Hierarchical classification (info file = taxonomy tree)
	-i info: (additional info specified in a file)
	-m max_iter: maximum_outer_iteration
	-l lambda: regularization coefficient (default 1.0)
	-e epsilon: stop criterion (default 1e-6)

Predicting:
Usage: ./predict (options) [test_data] [model] (prediction_result)

options:
	-p problem_type: (default 1)
		1 -- Sequence labeling (info file = feature template)
		2 -- Hierarchical classification (info file = taxonomy tree)
	-i info: (additional info specified in a file)

Please see README attached in the tarball for more details.


Examples

Training for sequence labeling:
	$ ./train -p 1 -i ../example/seq_label/seq_label_feature_template ../example/seq_label/ocr.deg2.100

Predicting for sequence labeling:
	$  ./predict -p 1 -i ../example/seq_label/seq_label_feature_template ../example/seq_label/ocr.deg2.t49 model prediction_result	

Training for hierarchical classification:
	$ ./train -p 2 -l 0.001 -i ../example/taxonomy/cat_hier.txt ../example/taxonomy/train.hier.20 model.hier

Predicting for hierarchical classification:
	$ ./predict -p 2 -i ../example/taxonomy/cat_hier.txt ../example/taxonomy/test.hier.5 model.hier prediction_result


Citation

You are welcome to use the code, however please acknowledge its use with a citation:

Proximal Quasi-Newton for Computationally Intensive l1-regularized M-estimators,
Kai Zhong, Ian E.H. Yen, Inderjit S. Dhillon and Pradeep Ravikumar,
Neural Information Processing Systems (NIPS), December 2014, [PDF]

If you just want to repeat the experiment in the paper, please download this code, which also includes other optimizers. Please see README file in the tarball for how to repeat the experiment in the paper.

Bug reports and comments are always appreciated.