Scale By the Bay 2020

Natural language understanding is a key component in many data science systems that must understand or reason about text. Common use cases include information extraction, summarization, sentiment analysis, document classification, language modeling, and disambiguation. This talk introduces the Spark NLP library - the world's most widely used NLP library in the enterprise. Accuracy, speed, and scalability benchmarks and design best practices for building NLP, ML and DL pipelines will be shared. The library implements core NLP algorithms including lemmatization, part of speech tagging, dependency parsing, named entity recognition, spell checking, document classification, and sentiment detection. The talk will demonstrate using these algorithms to build commonly used pipelines, using live Python notebooks that will be made publicly available after the talk.