Separate tokenizer stuff into parsing and configuration types.