About the Book

This book is designed for beginners to data analysis and covers the basics of Python data analysis programming and statistics. It covers the Python fundamentals that are necessary to data analysis, including objects, functions, modules and libraries. The libraries that are integral to data science are explored and explained, including NumPy, SciPy, BeautifulSoup, Pandas and MatPlobLib.

·Getting Started with Python for Data Science
·Getting Your Hands Dirty with Data
·Visualizing the Invisible
·Wrangling Data
·Learning from Data
·The Part of Tens

About the Author

John Paul Mueller, consultant, application developer, writer and technical editor, has written over 600 articles and 97 books. His topics range from programming to home security. Luca Massaron is a data scientist and a research director specializing in multivariate statistical analysis, machine learning and customer insight. He is a pioneer of Web audience analysis in Italy and was named one of the top ten data scientists at competitions by kaggle.com.

Table of Contents:
Introduction About This Book Foolish Assumptions Icons Used in This Book Beyond the Book Where to Go from Here Part I: Getting Started with Python for Data Science Chapter 1: Discovering the Match between Data Science and Python ·Defining the Sexiest Job of the 21st Century ·Considering the emergence of data science ·Outlining the core competencies of a data scientist ·Linking data science and big data ·Understanding the role of programming ·Creating the Data Science Pipeline ·Preparing the data ·Performing exploratory data analysis ·Learning from data ·Visualizing ·Obtaining insights and data products ·Understanding Python's Role in Data Science ·Considering the shifting profile of data scientists ·Working with a multipurpose, simple and efficient language ·Learning to Use Python Fast ·Loading data ·Training a model ·Viewing a result Chapter 2: Introducing Python's Capabilities and Wonders ·Why Python? ·Grasping Python's core philosophy ·Discovering present and future development goals ·Working with Python ·Getting a taste of the language ·Understanding the need for indentation ·Working at the command line or in the IDE ·Performing Rapid Prototyping and Experimentation ·Considering Speed of Execution ·Visualizing Power ·Using the Python Ecosystem for Data Science ·Accessing scientific tools using SciPy ·Performing fundamental scientific computing using NumPy ·Performing data analysis using pandas ·Implementing machine learning using Scikit learn ·Plotting the data using matplotlib ·Parsing HTML documents using Beautiful Soup Chapter 3: Setting Up Python for Data Science ·Considering the Off the Shelf Cross Platform Scientific Distributions ·Getting Continuum Analytics Anaconda ·Getting Enthought Canopy Express ·Getting pythonxy ·Getting WinPython ·Installing Anaconda on Windows ·Installing Anaconda on Linux ·Installing Anaconda on Mac OS X ·Downloading the Datasets and Example Code ·Using IPython Notebook ·Defining the code repository ·Understanding the datasets used in this book Chapter 4: Reviewing Basic Python ·Working with Numbers and Logic ·Performing variable assignments ·Doing arithmetic ·Comparing data using Boolean expressions ·Creating and Using Strings ·Interacting with Dates ·Creating and Using Functions ·Creating reusable functions ·Calling functions in a variety of ways ·Using Conditional and Loop Statements ·Making decisions using the if statement ·Choosing between multiple options using nested decisions ·Performing repetitive tasks using for ·Using the while statement ·Storing Data Using Sets, Lists and Tuples ·Performing operations on sets ·Working with lists ·Creating and using Tuples ·Defining Useful Iterators ·Indexing Data Using Dictionaries Part II: Getting Your Hands Dirty with Data Chapter 5: Working with Real Data ·Uploading, Streaming and Sampling Data ·Uploading small amounts of data into memory ·Streaming large amounts of data into memory ·Sampling data ·Accessing Data in Structured Flat File Form ·Reading from a text file ·Reading CSV delimited format ·Reading Excel and other Microsoft Office files ·Sending Data in Unstructured File Form ·Managing Data from Relational Databases ·Interacting with Data from NoSQL Databases ·Accessing Data from the Web Chapter 6: Conditioning Your Data ·Juggling between NumPy and pandas ·Knowing when to use NumPy ·Knowing when to use pandas ·Validating Your Data ·Figuring out what's in your data ·Removing duplicates ·Creating a data map and data plan ·Manipulating Categorical Variables ·Creating categorical variables ·Renaming levels ·Combining levels ·Dealing with Dates in Your Data ·Formatting date and time values ·Using the right time transformation ·Dealing with Missing Data ·Finding the missing data ·Encoding missingness ·Imputing missing data ·Slicing and Dicing and Selecting Data ·Slicing rows ·Slicing columns ·Dicing ·Concatenating and Transforming ·Adding new cases and variables ·Removing data ·Sorting and shuffling ·Aggregating Data at Any Level Chapter 7: Shaping Data ·Working with HTML Pages ·Parsing XML and HTML ·Using XPath for data extraction ·Working with Raw Text ·Dealing with Unicode ·Stemming and removing stop words ·Introducing regular expressions ·Using the Bag of Words Model and Beyond ·Understanding the bag of words model ·Working with ngrams ·Implementing TFIDF transformations ·Working with Graph Data ·Understanding the adjacency matrix ·Using NetworkX basics Chapter 8: Putting What You Know in Action ·Contextualizing Problems and Data ·Evaluating a data science problem ·Researching solutions ·Formulating a hypothesis ·Preparing your data ·Considering the Art of Feature Creation ·Defining feature creation ·Combining variables ·Understanding binning and discretization ·Using indicator variables ·Transforming distributions ·Performing Operations on Arrays ·Using vectorization ·Performing simple arithmetic on vectors and matrices ·Performing matrix vector multiplication ·Performing matrix multiplication Part III: Visualizing the Invisible Chapter 9: Getting a Crash Course in MatPlotLib ·Starting with a Graph ·Defining the plot ·Drawing multiple lines and plots ·Saving your work ·Setting the Axis, Ticks, Grids ·Getting the axes ·Formatting the axes ·Adding grids ·Defining the Line Appearance ·Working with line styles ·Using colors ·Adding markers ·Using Labels, Annotations and Legends ·Adding labels ·Annotating the chart ·Creating a legend Chapter 10: Visualizing the Data ·Choosing the Right Graph ·Showing parts of a whole with pie charts ·Creating comparisons with bar charts ·Showing distributions using histograms ·Depicting groups using box plots ·Seeing data patterns using scatterplots ·Creating Advanced Scatterplots ·Depicting groups ·Showing correlations ·Plotting Time Series ·Representing time on axes ·Plotting trends over time ·Plotting Geographical Data ·Visualizing Graphs ·Developing undirected graphs ·Developing directed graphs Chapter 11: Understanding the Tools ·Using the IPython Console ·Interacting with screen text ·Changing the window appearance ·Getting Python help ·Getting IPython help ·Using magic functions ·Discovering objects ·Using IPython Notebook ·Working with styles ·Restarting the kernel ·Restoring a checkpoint ·Performing Multimedia and Graphic Integration ·Embedding plots and other images ·Loading examples from online sites ·Obtaining online graphics and multimedia Part IV: Wrangling Data Chapter 12: Stretching Python's Capabilities ·Playing with Scikit learn ·Understanding classes in Scikit learn ·Defining applications for data science ·Performing the Hashing Trick ·Using hash functions ·Demonstrating the hashing trick ·Working with deterministic selection ·Considering Timing and Performance ·Benchmarking with time it ·Working with the memory profiler ·Running in Parallel ·Performing multicore parallelism ·Demonstrating multiprocessing Chapter 13: Exploring Data Analysis ·The EDA Approach ·Defining Descriptive Statistics for Numeric Data ·Measuring central tendency ·Measuring variance and range ·Working with percentiles ·Defining measures of normality ·Counting for Categorical Data ·Understanding frequencies ·Creating contingency tables ·Creating Applied Visualization for EDA ·Inspecting boxplots ·Performing to tests after boxplots ·Observing parallel coordinates ·Graphing distributions ·Plotting scatterplots ·Understanding Correlation ·Using covariance and correlation ·Using nonparametric correlation ·Considering chi square for tables ·Modifying Data Distributions ·Using the normal distribution ·Creating a Z score standardization ·Transforming other notable distributions Chapter 14: Reducing Dimensionality ·Understanding SVD ·Looking for dimensionality reduction ·Using SVD to measure the invisible ·Performing Factor and Principal Component Analysis ·Considering the psychometric model ·Looking for hidden factors ·Using components, not factors ·Achieving dimensionality reduction ·Understanding Some Applications ·Recognizing faces with PCA ·Extracting Topics with NMF ·Recommending movies Chapter 15: Clustering ·Clustering with K means ·Understanding centroid based algorithms ·Creating an example with image data ·Looking for optimal solutions ·Clustering big data ·Performing Hierarchical Clustering ·Moving Beyond the Round-Shaped Clusters: DBScan Chapter 16: Detecting Outliers in Data ·Considering Detection of Outliers ·Finding more things that can go wrong ·Understanding anomalies and novel data ·Examining a Simple Univariate Method ·Leveraging on the Gaussian distribution ·Making assumptions and checking out ·Developing a Multivariate Approach ·Using principal component analysis ·Using cluster analysis ·Automating outliers detection with SVM Part V: Learning from Data Chapter 17: Exploring Four Simple and Effective Algorithms ·Guessing the Number: Linear Regression ·Defining the family of linear models ·Using more variables ·Understanding limitations and problems ·Moving to Logistic Regression ·Applying logistic regression ·Considering when classes are more ·Making Things as Simple as Naïve Bayes ·Finding out that Naïve Bayes isn't so naïve ·Predicting text classifications ·Learning Lazily with Nearest Neighbors ·Predicting after observing neighbors ·Choosing your k parameter wisely Chapter 18: Performing Cross Validation, Selection and Optimization ·Pondering the Problem of Fitting a Model ·Understanding bias and variance ·Defining a strategy for picking models ·Dividing between training and test sets ·Cross Validating ·Using cross validation on k folds ·Sampling stratifications for complex data ·Selecting Variables Like a Pro ·Selecting by univariate measures ·Using a greedy search ·Pumping Up Your Hyper parameters ·Implementing a grid search ·Trying a randomized search Chapter 19: Increasing Complexity with Linear and Nonlinear Tricks ·Using Nonlinear Transformations ·Doing variable transformations ·Creating interactions between variables ·Regularizing Linear Models ·Relying on Ridge regression (L2) ·Using the Lasso (L1) ·Leveraging regularization ·Combining L1 & L2: Elasticnet ·Fighting with Big Data Chunk by Chunk ·Determining when there is too much data ·Implementing Stochastic Gradient Descent ·Understanding Support Vector Machines ·Relying on a computational method ·Fixing many new parameters ·Classifying with SVC ·Going nonlinear is easy ·Performing regression with SVR ·Creating a stochastic solution with SVM Chapter 20: Understanding the Power of the Many ·Starting with a Plain Decision Tree ·Understanding a decision tree ·Creating classification and regression trees ·Making Machine Learning Accessible ·Working with a Random Forest classifier ·Working with a Random Forest regressor ·Optimizing a Random Forest ·Boosting Predictions ·Knowing that many weak predictors win ·Creating a gradient boosting classifier ·Creating a gradient boosting regressor ·Using GBM hyper parameters Part VI: The Part of Tens Chapter 21: Ten Essential Data Science Resource Collections ·Gaining Insights with Data Science Weekly ·Obtaining a Resource List at U Climb Higher ·Getting a Good Start with KDnuggets ·Accessing the Huge List of Resources on Data Science Central ·Obtaining the Facts of Open Source Data Science from Masters ·Locating Free Learning Resources with Quora ·Receiving Help with Advanced Topics at Conductrics ·Learning New Tricks from the Aspirational Data Scientist ·Finding Data Intelligence and Analytics Resources at Analytic Bridge ·Zeroing In on Developer Resources with Jonathan Bower Chapter 22: Ten Data Challenges You Should Take ·Meeting the Data Science London + Scikit learn Challenge ·Predicting Survival on the Titanic ·Finding a Kaggle Competition that Suits Your Needs ·Honing Your Overfit Strategies ·Trudging Through the MovieLens Dataset ·Getting Rid of Spam Emails ·Working with Handwritten Information ·Working with Pictures ·Analyzing Amazon.com Reviews ·Interacting with a Huge Graph Index