Tag: nextToken

StreamTokenizer Tokenize Stream

April 18, 2011 · 3 min read
Breaking a string or stream into meaningful independent words is known as **tokenization**. Tokenization is a common practice to tool developers. **java.util** package includes **[StringTokenizer](http://way2java.com/java-util/class-stringtokenizer/)** which tokenizes a string into independent words. For StringTokenizer, the source is a string. There comes a similar tokenizer, **StreamTokenize** with **java.io** package for which source is a stream. Here, the StreamTokenizer tokenizes a whole stream into independent words.

StringTokenizer Tokenizing String

March 28, 2011 · 2 min read
After knowing the data structures from **java.util** package, let us discuss about the remaining classes of **java.util** package. The classes include **Date** and **GregorianCalendar** to manipulate dates, **Random** class to generate random numbers and **StringTokenizer** to separate the string into individual words (known as tokens). The other classes less used are **Observer**, **Observable**, **Timer** and **TimerTask** etc. Now let us go through these classes one after another.