tokenize
Ignore parentheses with string tokenizer?
I have an input that looks like: (0 0 0) I would like to ignore the parenthesis and only add the numbers, in this case 0, to an arraylist.[详细]
2023-01-21 20:17 分类:问答Why is my string parsed differently via strtok on Windows and Linux?
In my program I\'m cutting my char* with strtok.When I\'m checking on Windows it\'s cut like I want, but when I\'m doing the same thing on Linux, it\'s doing it wrong.[详细]
2023-01-17 21:31 分类:问答Indexing n-word expressions as a single term in Lucene
I want to index a \"compound word\" like \"New York\" as a single term in Lucene not like \"new\", \"york\". In such a way that if someone searches for \"new place\", documents containing \"new york\"[详细]
2023-01-17 07:24 分类:问答MALLET tokenizer
Hi I want to use MALLET\'s topic modeling but can i provide my own tokenizer or tokenized version o开发者_高级运维f the text documents when i import the data into mallet? I find MALLET\'s tokenizer in[详细]
2023-01-16 18:41 分类:问答Split column to multiple rows
I have table with a column that contains multiple values separated by comma (,) and would like to split it so I get earch Site on its own row but with the same Number in front.[详细]
2023-01-16 08:13 分类:问答New to ASP, trying to use a tokenizer
Sorry for the kind of noob question but having is开发者_Python百科sues trying to get a Tokenizer working. Tried this example but on the line of the Tokenize() I get an error Type mismatched. I\'ve als[详细]
2023-01-16 02:14 分类:问答Question regarding regex and tokenizing
I need to make a tokenizer that is able to English words. Currently, I\'m stuck with characters where they can be part of of a url expression.[详细]
2023-01-16 00:46 分类:问答sqlite-fts3: custom tokenizer?
Does anyone here have experience with writing custom FT开发者_开发问答S3 (the full-text-search extension) tokenizers? I\'m looking for a tokenizer that will ignore HTML tags.[详细]
2023-01-15 16:49 分类:问答Lucene.NET: Camel case tokenizer?
I\'ve started playing with Lucene.NET today and I wrote a simple test method to do indexing and searching on source code files. The problem is that the standard analyze开发者_如何学Gors/tokenizers tre[详细]
2023-01-15 10:22 分类:问答Is there a Javascript lexer / tokenizer (in PHP)?
I\'ve seen a couple of Python Javascript tokenizers and a c开发者_如何学JAVAryptic document on Mozilla.org about a Javascript Lexer but can\'t find any Javascript tokenizers for PHP specifically. Are[详细]
2023-01-13 07:08 分类:问答