From news and speeches to informal chatter on social media, natural language is one of the richest and most underutilized sources of data. Not only does it come in a constant stream, always changing and adapting in context; it also contains information that is not conveyed by traditional data sources. The key to unlocking natural language is through the creative application of text analytics. This practical book presents a data scientist’s approach to building language-aware products with applied machine learning.

You’ll learn robust, repeatable, and scalable techniques for text analysis with Python, including contextual and linguistic feature engineering, vectorization, classification, topic modeling, entity resolution, graph analysis, and visual steering. By the end of the book, you’ll be equipped with practical methods to solve any number of complex real-world problems.

• Preprocess and vectorize text into high-dimensional feature representations
• Perform document classification and topic modeling
• Steer the model selection process with visual diagnostics
• Extract key phrases, named entities, and graph structures to reason about data in text
• Build a dialog framework to enable chatbots and language-driven interaction
• Use Spark to scale processing power and neural networks to scale model complexity

Ready to use statistical and machine-learning techniques across large data sets? This practical guide shows you why the Hadoop ecosystem is perfect for the job. Instead of deployment, operations, or software development usually associated with distributed computing, you’ll focus on particular analyses you can build, the data warehousing techniques that Hadoop provides, and higher order data workflows this framework can produce.

Data scientists and analysts will learn how to perform a wide range of techniques, from writing MapReduce and Spark applications with Python to using advanced modeling and data management with Spark MLlib, Hive, and HBase. You’ll also learn about the analytical processes and data systems available to build and empower data products that can handle—and actually require—huge amounts of data.

• Understand core concepts behind Hadoop and cluster computing
• Use design patterns and parallel analytical algorithms to create distributed data analysis jobs
• Learn about data management, mining, and warehousing in a distributed context using Apache Hive and HBase
• Use Sqoop and Apache Flume to ingest data from relational databases
• Program complex Hadoop and Spark applications with Apache Pig and Spark DataFrames
• Perform machine learning techniques such as classification, clustering, and collaborative filtering with Spark’s MLlib