Generating tokens in Python [duplicate]

This question already has answers here: Closed 11 years ago.

Possible Duplicate:

Is there a good Python library that can parse C++?

I want Python to scan through a file (a .cpp file) and generate tokens from it using the in-built Python tokeniser.How can thi开发者_如何学JAVAs be achieved?

---

The built-in tokenizer and ast stuff is for parsing Python, not other languages like C++.

You may want to look at GCC-XML.