“Python nltk tokenize” Code-Antworten

Python nltk tokenize

>>> import nltk
>>> sentence = """At eight o'clock on Thursday morning
... Arthur didn't feel very good."""
>>> tokens = nltk.word_tokenize(sentence)
>>> tokens
['At', 'eight', "o'clock", 'on', 'Thursday', 'morning',
'Arthur', 'did', "n't", 'feel', 'very', 'good', '.']
Tame Trout

Word_Tokenize importieren

import nltk
from nltk import word_tokenize
Itchy Impala

nltk python wie man Text tokenisieren

>>> tokens = word_tokenize(raw)
>>> type(tokens)
<class 'list'>
>>> len(tokens)
254354
>>> tokens[:10]
['The', 'Project', 'Gutenberg', 'EBook', 'of', 'Crime', 'and', 'Punishment', ',', 'by']
GelatinousMustard

Ähnliche Antworten wie “Python nltk tokenize”

Fragen ähnlich wie “Python nltk tokenize”

Weitere verwandte Antworten zu “Python nltk tokenize” auf Python

Durchsuchen Sie beliebte Code-Antworten nach Sprache

Durchsuchen Sie andere Codesprachen

Shell/Bash
C#
C++
C
CSS
HTML
Java
JavaScript
Objective-C
PHP
Python
Sql
Swift
Ruby
TypeScript
Go
Kotlin
Assembly
R
VBA
Scala
Rust
Dart
Elixir
Clojure
Haskell
Matlab
Erlang
Cobol
Fortran
Scheme
Perl
Groovy
Lua
Julia
Delphi
Abap
Lisp
Prolog
Pascal
ActionScript
Basic
Solidity
PowerShell
GDScript
Excel