black_dream

Hi everybody

Can anyone help me to write a code in python. This code will be able to read a text file and output a list of sentences, each represented as a list of tokens. The output should be :
-- The number of tokens in the text file
-- The length of each sentence (length of sentence = #of tokens in a sentence)

It seems a very simple code. But I am beginner in python and I spent 2 days in order to learn it. :eek:

So, please help me as much as you can. :o

Moldz

What have you got so far?

black_dream

The main problem is the reading from (a text file).
so, for example, I want to know
How can I read from a file (i.e. line by line)?
How can I count each word separately?
At the same time, how can I count the # of sentences and words in each sentence?

We know that each sentence ends with (e.g. "." , "?" , "!" , .... etc).

These are my assumptions, but I don't know how to implement them.

black_dream

can anybody help me ??????

Berto

if they are your assumptions then you can look up how to do it on google and attepmt it first?

Moldz

What you need to do is not difficult, especially in python. The problem here is that it sounds too much like a homework assignment and it doesn't seem like you've tried writing any code.

The functions you need are very basic and would be explained in almost any beginner's tutorial. Your assumptions are right on point, so look some stuff up. Start by finding info on the open() function (in the python shell, type "help(open)"). Also, try looking through the string functions ("help(str)"). Write down some code and experiment. If you're confused about how a function works, post a question here.

black_dream

So far, I am able to read from a text file.
I don't want a complete code. what I really want are some hints so I can do it easily.
It is not a homework assignment as (Moldz) mentioned. It is a kind of exercise I try to do it. If you can help me or it doesn't matter and thank you for replying.


--black_dream

Moldz

Well then, check out the split(), index() or find() functions to break up the string into parts.

Berto

if you are reading in each word as you go check that the current word does not contain the end line symbol if it does then you knwo you are at the end of your line, and just slice the string taking off the last character to get the last word.

black_dream

For example,
"I am trying to write a code in python."

This sentence contains 9 tokens which is also the length of the sentence. Also notice that that sentence ends with "."

Is there any (tokenizer methods) in python like in java and if there is, how can be implemented.

--black_dream