This Book is Available on BuyUBooks.
com
Ashley Davis
MANNING
[Link]
This Book is Available on [Link]
Data Wrangling with JavaScript
[Link]
This Book is Available on [Link]
[Link]
This Book is Available on [Link]
Data Wrangling
with JavaScript
ASHLEY DAVIS
MANNING
Shelter Island
[Link]
This Book is Available on [Link]
For online information and ordering of this and other Manning books, please visit [Link].
The publisher offers discounts on this book when ordered in quantity.
For more information, please contact
Special Sales Department
Manning Publications Co.
20 Baldwin Road
PO Box 761
Shelter Island, NY 11964
Email: orders@[Link]
©2019 by Manning Publications Co. All rights reserved.
No part of this publication may be reproduced, stored in a retrieval system, or transmitted, in any form
or by means electronic, mechanical, photocopying, or otherwise, without prior written permission of the
publisher.
Many of the designations used by manufacturers and sellers to distinguish their products are claimed
as trademarks. Where those designations appear in the book, and Manning Publications was aware of a
trademark claim, the designations have been printed in initial caps or all caps.
∞ Recognizing the importance of preserving what has been written, it is Manning’s policy to have the books
we publish printed on acid-free paper, and we exert our best efforts to that end. Recognizing also our
responsibility to conserve the resources of our planet, Manning books are printed on paper that is at
least 15 percent recycled and processed without the use of elemental chlorine.
Manning Publications Co. Development editor: Helen Stergius
20 Baldwin Road Technical development editor: Luis Atencio
PO Box 761 Review editor: Ivan Martinovic´
Shelter Island, NY 11964 Project manager: Deirdre Hiam
Copy editor: Katie Petito
Proofreader: Charles Hutchinson
Technical proofreader: Kathleen Estrada
Typesetting: Happenstance Type-O-Rama
Cover designer: Marija Tudor
ISBN 9781617294846
Printed in the United States of America
1 2 3 4 5 6 7 8 9 10 – SP – 23 22 21 20 19 18
[Link]
This Book is Available on [Link]
brief contents
1 ■ Getting started: establishing your data pipeline 1
2 ■ Getting started with [Link] 25
3 ■ Acquisition, storage, and retrieval 59
4 ■ Working with unusual data 99
5 ■ Exploratory coding 115
6 ■ Clean and prepare 143
7 ■ Dealing with huge data files 168
8 ■ Working with a mountain of data 191
9 ■ Practical data analysis 217
10 ■ Browser-based visualization 247
11 ■ Server-side visualization 274
12 ■ Live data 299
13 ■ Advanced visualization with D3 329
14 ■ Getting to production 358
[Link]
This Book is Available on [Link]
[Link]
This Book is Available on [Link]
contents
preface xv
acknowledgments xvii
about this book xix
about the author xxiii
about the cover illustration xxv
1 Getting started: establishing your data pipeline
1.1 Why data wrangling? 1
1
1.2 What’s data wrangling? 2
1.3 Why a book on JavaScript data wrangling? 3
1.4 What will you get out of this book? 4
1.5 Why use JavaScript for data wrangling? 5
1.6 Is JavaScript appropriate for data analysis? 6
1.7 Navigating the JavaScript ecosystem 7
1.8 Assembling your toolkit 7
1.9 Establishing your data pipeline 8
Setting the stage 9 The data-wrangling
■
process 10 Planning 10 Acquisition, storage, and
■ ■
retrieval 13 Exploratory coding 15 Clean and
■ ■
prepare 18 Analysis 19 Visualization 20 Getting
■ ■ ■
to production 22
vii
[Link]
viii This Book is Available on [Link]
CONTENTS
2 Getting started with [Link]
2.1 Starting your toolkit 26
25
2.2 Building a simple reporting system 27
2.3 Getting the code and data 27
Viewing the code 28 Downloading the code 28 Installing
■ ■
[Link] 29 Installing dependencies 29 Running
■ ■
[Link] code 29 Running a web application 30 Getting
■ ■
the data 30 Getting the code for chapter 2 31
■
2.4 Installing [Link] 31
Checking your [Link] version 32
2.5 Working with [Link] 33
Creating a [Link] project 33 Creating a command-line
■
application 36 Creating a code library 38 Creating a
■ ■
simple web server 40
2.6 Asynchronous coding 45
Loading a single file 46 Loading multiple ■
files 49 Error handling 51 Asynchronous coding
■ ■
with promises 52 Wrapping asynchronous operations in
■
promises 55 Async coding with “async” and “await” 57
■
3 Acquisition, storage, and retrieval
3.1 Building out your toolkit 60
59
3.2 Getting the code and data 61
3.3 The core data representation 61
The earthquakes website 62 ■
Data formats covered 64
Power and flexibility 65
3.4 Importing data 66
Loading data from text files 66 Loading data from a REST
■
API 69 Parsing JSON text data 70 Parsing CSV text
■ ■
data 74 Importing data from databases 78 Importing
■ ■
data from MongoDB 78 Importing data from MySQL 82
■
3.5 Exporting data 85
You need data to export! 85 Exporting data to text
■
files 85 Exporting data to JSON text files 87
■
Exporting data to CSV text files 89 Exporting data ■
to a database 90 Exporting data to MongoDB 91
■
Exporting data to MySQL 92
[Link]
This Book is Available on [Link]
CONTENTS ix
3.6 Building complete data conversions 95
3.7 Expanding the process 95
4 Working with unusual data
4.1 Getting the code and data
99
100
4.2 Importing custom data from text files 101
4.3 Importing data by scraping web pages 104
Identifying the data to scrape 104 ■
Scraping with Cheerio 105
4.4 Working with binary data 107
Unpacking a custom binary file 108 Packing a custom binary
■
file 111 Replacing JSON with BSON 113 Converting JSON
■ ■
to BSON 113 Deserializing a BSON file 114
■
5 Exploratory coding
5.1
115
Expanding your toolkit 116
5.2 Analyzing car accidents 116
5.3 Getting the code and data 117
5.4 Iteration and your feedback loop 117
5.5 A first pass at understanding your data 118
5.6 Working with a reduced data sample 120
5.7 Prototyping with Excel 120
5.8 Exploratory coding with [Link] 122
Using Nodemon 123 Exploring your data 125
■
Using Data-Forge 128 Computing the trend
■
column 130 Outputting a new CSV file 134
■
5.9 Exploratory coding in the browser 135
5.10 Putting it all together 141
6 Clean and prepare
6.1
143
Expanding our toolkit 144
6.2 Preparing the reef data 145
6.3 Getting the code and data 145
6.4 The need for data cleanup and preparation 145
6.5 Where does broken data come from? 145
[Link]
