Function rnix::tokenizer::tokenize

source ยท 
pub fn tokenize(input: &str) -> Vec<Token<'_>>
Expand description

A convenience function for tokenizing the given input