Python Data Science for Kids Taster Workshops

During May and June I ran a series of taster workshops in several locations in Cornwall for children aged 7-17 designed to:

• introduce some of these standard tools to children aged 7-17
• provide some experience of methods like data loading, cleaning, visualisation, exploring, machine learning

The event page is here:

• https://sites.google.com/view/python-datascience-for-kids/

Data Science and Python

Data Science is a bread term which covers a range is valuable skills - from coding to machine learning, from data engineering to visualisation.

Python has become the leading tool for data scientists by far - and some of tools in the Python ecosystem are not just defacto standards, but familiarity with them is pretty much expected. These standard tools include the jupyter notebook and libraries like pandas and scikit-learn.

I think it is incredibly advantageous for children to have some experience with these tools, and I think it is important for them to practice some of the data science disciplines, such as data cleaning and visualisation.

Taster Workshops for Kids

The series of workshops was supported by a grant from Numfocus and the Jupyter Project. NumFocus is acharity whose mission is to promote open practices in research, data, and scientific computing. They support many of the open source data science tools you very likely already use. You can read a blog announcing the supported projects here:

• https://blog.jupyter.org/jupyter-community-workshops-a7f1dca1735e

Mini Projects For Kids

It is always a challenge to create activities for children that are engaging and also meaningfully help children learn something new. 

Activities need to be small enough so they don't overwhelm, and of a duration that matches a child's comfortable attention span. 

It helps if the activities can be i the form of a story - to make the ideas more real and relatable. 

Furthermore, in single workshops there is limited scope to take children through a lot of pre-requisite training in Python - so the activities need to have a lot of the boiler-plate work removed or already done. This means a carefully thought out balance between "pre-typed code" and instructions and questions which as a child to experiment and explore, or solve a puzzle. 

In my own experience, it helps to avoid any kind of technical complexity like installing and configuring software. Web-based tools that require no installation work best in the limited time, attention and diverse setting of a children's workshop.

With this in mind I came up with a series of projects at different levels of difficulty, all using Google's hosted colab notebook service.

Demonstrating Python and the Jupyter Notebook

At the start of each workshop I talked briefly about the importance of data science at a global scale as well as its relevance to Cornwall.

I then demonstrated basic Python and the Jupyter notebook to show how it works, and to illustrate how easy coding with Python is. I showed how the notebook is just a web page with fields to fill in and run using the "play" button. Having no need to install any software and configure it was a major relief!

The basic python was simply variables, print statements, progressing onto using a list of children's ages, and using operations on the list like max() and len(). The lack of a mean() or average() was nice point to show that it is common to pull in extension libraries that implement features not part of the core Python. I showed how to import pandas, and demonstrated the dataframe, which does have a mean() function. I then showed how easy it was to plot a dataframe as a linechart, and change it to a bar chart, and then a histogram.

I emphasised the important point that learning all the instructions of a language or its libraries is not the aim. A more important skill is being able to search the documentation and reference sites to find how Python and its libraries can be used to achieve your task.

0 - Getting Started

This short worksheet helps children and their parents or carers get set up to use the Google hosted notebook service.

It makes sure they have a Google account, and helps them create one if needed, and tests access to a simple hosted notebook to check everything is working.

• Getting Started (PDF)

1 - Hands And Fingers

This is a project suitable for younger children. It focuses on measuring the length of fingers on each hand and collecting that data.

The idea of a DataFrame is introduced, and these are used to plot charts showing the lengths. Very simple statistics are explored - the minimum, maximum and mean of a column of data. Children are encouraged to explore how their left and right hands are different using the statistics, but also see how it is much easier to see when the data is visualised.

The following photo shows a bar chart comparing the lengths of left and right hand using different colours for each hand.

The Hand and Fingers project and printable rulers are online:

• Hand And Fingers (PDF)
• Printable rulers (PDF)

2 - Garden Bug Detective

The next project starts simple and is set in a friendly story about a robot that collects items from the garden.

The robot doesn't know what it has picked up. It only knows how to measure the width, length and weight of the items.

The children are encouraged to visualise the data to get a high level view of it before diving into any further exploration. This time the first chart isn't very enlightening.

The idea of a histogram is introduced to see the data in a different way. The following photo shows a girl exploring a histogram which clearly shows that the data seems to have two groups - a good start to further exploration.

One child worked out how to show three data series on the same histogram chart!

Scatter charts were introduced next, and this visualisation revealed three definite clusters in the data.

With all these visualisations, the children were encouraged to vary what was plotted, and to use a search engine to find out what the code syntax should be.

The project then progresses to use the sklearn library to perform k-means clustering on the data.  The children were excited to be using the same software used by grown-up machine learning and AI researchers!

Seeing the computer identify the group clusters was exciting, and even more exciting was providing the trained model with new data to classify.

I felt it was important for the children to have seen this training and classification process at least once at first hand. I think it will place them in good stead when they consider or see machine learning again in future.

It was great seeing children as young at eight using sklearn to train a model, and use it to predict whether a garden item was a word, ladybird or stone!

The project is online:

• Garden Bug Detective (PDF)

2a - Secret Spy Messages

The next project focussed again on an engaging story to wrap an interesting data science concept.

One spy, Jane, is trying to get messages to another spy, John, but the messages arrived messed up by noise, probably caused baddie. Jane tries to send the message 20 times.

The children were asked to look at 20 messages to see if they can spot the hidden message. The photo below shows a child looking at these noisy images.

This project introduces images as data, and encourages the children to explore mathematical or other operations on images.  The project also demonstrates getting data through a URL and opening the received zip file.

The matplotlib library is extensively used to show bitmap images, which are 2d numpy arrays.

After the students try subtracting images, and failing, clues encourage them to add images. All the students discovered that adding more and more images seemed to reveal an image.

After that revelation, which seemed to excite the children, they were encouraged to think about why adding noisy images together seems to work.

The children, and especially the parents, found it very exciting to see a theoretical idea - average value of random noise being zero - applied in this useful and practical way.

The project is online:

• Secret Spy Messages (PDF)

3 - Mysterious Space X-Rays

The next project is a significant challenge for the more confident, enthusiastic or able children.

It uses real data from a NASA space mission which measures radiation from space. Often the only way to identify objects in deep space is to look at the only thing that gets to us on Earth - radiation.

The Cygnus X-3 system is a mysterious object which behaves in ways which aren't like the standard kinds of stars or other space objects.

The children are encouraged to explore the data and use any idea they have to extract any insightful pattern from the data. Both the children and parents found it exciting that this task was genuinely at the cutting edge of human understanding, and that any idea they had stood a chance of making them famous!

The project itself started by describing steps to look at and identify anomalous data, and then take data cleansing steps. After that, it intentionally stopped prescribing analysis steps, encouraging the children to think up and try their own ideas, using an internet search engine to read about those ideas and how they might be implemented in code. I emphasised again that this skill is valuable.

I was pleasantly surprised by the great ideas that some of the students came up with - including removing small amplitudes as a way of removing noise, or only keeping the very peaks of the data as a way to keep "radiation events".

Overall,  the more confident and able students really enjoyed working on a data challenge where there was no correct single answer. It was a huge contrast to the tasks they're set at school where there is only one correct answer, and an answer that has been found endlessly before.

The project is online:

• Mysterious Space X-Rays (PDF)

Conclusion & Thanks

The motivation behind this touring series of taster workshops was to give children actual experience of using the same tools that are used by professionals across the globe, doing exciting and cutting edge work from AI to data journalism. I also wanted the children to practice some of the methods and discipline from data science, such as visualising data to understand it better, data cleaning, and using different forms of visualisation to gain deeper insights.

A lesson that I learned was that a small number of children didn't follow the prompts to try things themselves or to think about solving some of the puzzles along the way. They were set deliberately because learning happens best when it is done actively rather than passively. I'm not sure there is a good solution to this that can work within the scope of a workshop - attitudes and values to learning come from a broader family environment.

I was really pleased to see some children found the projects genuinely exciting and left wanting to do more ... and I was rather surprised that the parents took as much interest in the projects as the children!

I'd like to thank all the groups that helped make this happen, including the Jupyter project, Numfocus, Carbubian Arts and Science Trust, the Royal Cornwall Museum, the Poly Falmouth, the Krowji Arts Centre and Falmouth University.

Python First Steps - A Hands On Tutorial

This month we had a first-steps introduction to Python. It was arranged in response to feedback from members who felt a beginners introduction would be useful in helping them explore the Python data science ecosystem of tools and methods.

The slides for the talk are online [link].

Aim

The aim of the session was not to provide a comprehensive coverage of python as a language, nor an exhaustive tour of the ecosystem of libraries and tools.

The aim was to:

• demonstrate enough of the basic of python to see how it works,
• write your own code,
• practice the important skill of searching the internet for code syntax and how to use libraries
• be able to understand a good amount of python code that others have written
• and most importantly, develop the confidence to continue to learn and explore python.

Throughout we emphasised the point that today, an encyclopaedic knowledge of a language or a library's options is not indicative of a good programmer. Today, languages and tools are so huge in number and size, that the ability to find the right tool and learn how to use it is a much more important skill. Added to this, the fact that tools change at an ever faster rate.

Why Python?

We briefly set the scene by looking at several recent charts showing Python as one of the fastest growing languages, already in the top 3 in most market analyses, and far ahead in the fields of data science and especially machine learning.

We pondered on the fact that python was not initially designed as a numerical language, but its ease of use accelerated its adoption on many fields including data science.

Notebooks

In the last decade a key innovation has emerged that has made coding easier, friendlier, and avoids the technical setup that was previously necessary.

That innovation is the notebook. In essence, it is just a web page into which we write our instructions, and see the results of those instructions.

A web page is already very familiar to many people and reduces the barriers to coding.

Today notebooks are both simple, and also very capable, with the ability to show charts, include animations, and even include control widgets.

Github, and other code repositories, even support previewing uploaded notebooks - here's an example from one of our own meetups:

• https://github.com/datasciencecornwall/top10_python_libraries/blob/master/top10_08_matplotlib.ipynb

Getting / Using Python

Most users of python make extensive use of the healthy and vibrant ecosystem of libraries and tools. The official python distribution from python.org is fairly capable but doesn't include many of the now popular libraries.

Many data scientists and machine learning researchers use the Anaconda Python distribution which includes many of the common libraries used in these fields. They are fairly well tested to work together, and the distribution even includes performance optimisations for Intel CPUs. Anaconda Python also includes the standard jupyter notebook system.

Another good alternative is to use Google's hosted system called colab. This makes using python even easier as there is nothing to install. Everything runs in Google's infrastructure, through a web browser. Despite being a test, the service is robust and growing rapidly in popularity. Even better, the service is free, subject to some controls to avoid exploitative use. Most compelling to machine learning researchers is free access to otherwise very expensive GPUs for accelerating computation.

Python Basics

We worked through the following key python concepts - first discussing them, then seeing some examples, and finally having a go at solving some of the challenges which were designed to test our understanding of the theme, or our ability to find answers on the internet.

• Variables and Lists
• Loops and Logic
• Functions
• Objects and Classes
• Visualisation

The slides include links to simple notebooks which you can open and explore, and even edit after you save your own copy. The following shows a snippet of the first notebook introducing variables and lists:

• https://colab.research.google.com/drive/1KMZI7XBsYp1BrajCoiFliBdo0p9u1Kku

The class did very well, working through all the themes. Noting that some had never coded before, this is quite impressive.

The only topic that caused some trouble was the more advanced topic of objects and classes, which could be a topic for an entire class itself.

Object oriented programming is considered an advanced topic, so it is still an achievement if attendees can recognise it in code they look at in future, even if not all the details are immediately clear.

Looking At Other's Code

To demonstrate that what we covered was indeed a large proportion of the basic elements from which real-world code is built, we looked at two different examples:

• a generative adversarial network which uses neural networks that learn to render faces
• a web application server which runs a twitter-like service

We noted that the machine learning code was built from now-familiar elements such as variables, functions, imports, loops, classes and objects and visualising numerical arrays.

The web application code was stark in how small it was - given the service was in essence the sae as twitter. The point of this was to show that, with libraries, many problems can be solved with a very small amount of python - and very understandable python at that.

Conclusion

Speaking with attendees afterwards, I was pleased that the session had:

• demystified coding and python
• given some the confidence to explore more, noting that what we covered in class is a large proportion of the foundations on which most code is built
• underlined the importance of research skills over memorising python instructions and options

Machine Learning for Image Classification - Tensorflow Tutorial

This month's meetup was a tutorial on machine learning to do image classification with Tensorflow.
We also had a short talk looking deeper at the last session's sentiment analysis.

Barney's image classification slides are at: (pdf). David's notebook on sentiment analysis is at: (link).

A video of the talks is at: (youtube).

A Deeper Look Sentiment Analysis

At the previous session we explored simple approaches to sentiment analysis, in particular the lexical approach of summing scores associated with words known to be positive or negative.

David dig deeper into that approach and found that documents given a positive or negative overall score actually have many positive and negative scored words within them. The following histogram illustrates this.

David's explorations remind us that it is important to:

• understand your data and not apply analysis blindly
• understand the limits or weakness of an algorithm
• a statistical answer isn't complete without a measure of "confidence"

You can find more if his code and results in his bitbucket.

Image Classification - Automating Manual Processes

Barney started in a very compelling scenario of a business manually sorting paper - invoices, cash claims, letters. The work itself is boring, very slow, prone to fatigue and error, and not good use of people's time.

A natural question occurred to him - could that manual process of classifying document be automated?

Barney looked to modern neural network based machine learning methods which have proven very successful at image classification.

This illustration shows a neural network learning to classify images from a data set as one of three particular characters.

Neural networks learn by adjusting link weights between nodes that make up layers of nodes. Given a training example, the error in its prediction is used to update link weights by a small amount to try to improve that prediction. Over many training examples, a neural network can get better and better at classifying a given image.

Although it is tempting to build a neural network from scratch, in many industrial applications it makes sense to use architectures that have been proven suitable for a given task. The neural network architecture for image classification will likely be different to one for natural language prediction.

Barney discussed Google's Inception network, a large network optimised for image classification.

There are some excellent articles online that explain the history and rationale of Google's Inception networks:

• Recent history of key image classification architectures, AlexNet, VGG16 and Inception: https://cv-tricks.com/cnn/understand-resnet-alexnet-vgg-inception/
• Notes on a reading on Google's Inception paper: https://www.analyticsvidhya.com/blog/2018/10/understanding-inception-network-from-scratch/
• Good overview of the design considerations of the various Inception network version is here: https://towardsdatascience.com/a-simple-guide-to-the-versions-of-the-inception-network-7fc52b863202

The deep (and wide) Inception network is trained on 1.2 million images, and is only practical with large large compute power available to organisations like Google.

Barney explained that we don't need to train such large complex networks from scratch - we can take advantage of the training that Google has done and simply extend that training to our own data. This is called transfer learning.

In essence the start (left part) of the network has learned to pick out features that help it with the task of image classification. We retain this learning and only train the small end of the network to focus on learning our own subset of images, making use of the same features learned from the huge training data.

You can read more about transfer learning here:

• https://medium.com/decathlondevelopers/how-you-can-build-an-image-classifier-in-one-day-part-1-transfer-learning-5602d3f96e6b

Barney's results were very promising and has generated significant excitement about automating the manual business process. He continues to develop and refine his solution.

Barney touched on an important aspect of automation - the impact on people and employment. His analysis is that businesses should focus on people shifting away from boring and low-skilled tasks towards more challenging and creative work - and the very same people previously employed can do this.

Tensorflow Walk-Through

Barney walked us through a python notebook which demonstrates the training of a simple network using the popular Tensorflow machine learning framework to classify images of fashion items from the MNIST fashion dataset.

The online colab notebook which you can run, and is very well commented, is here:

• https://colab.research.google.com/github/tensorflow/docs/blob/master/site/en/tutorials/keras/basic_classification.ipynb

The central elements of the process are:

• import Tensorflow and a higher-level API Keras which makes describing and using neural networks easier
• import the MNIST fashion data set of 60,000 training images and 10,000 test images
• convert the grey-scale image data from 0-255 to 0-1
• construct a 3-layer network, the input layer of which is 28x28 which match the image size:

• the network model is "compiled" with a loss-function and a method for adjusting the weights, commonly called gradient descent of which there are many options:

• the neural network is then trained (5 times, or epochs) using the 60,000 image training data set:

• after training, we check how well the network has been trained, by testing it on the 10,000 image test set:

That score of 0.8778 means the neural network correctly classified 88% of the 10,000 test images - an excellent initial result!

Barney did explore an important aspect of image classification. He first showed and example of an ankle boot and the result of a network prediction. It is clear that the network has a very clear and high belief the boot image is indeed of an ankle boot.

He then showed us a more interesting example. Here is the network confidently but incorrectly classifying a sneaker as a sandal. 

A different run shows the network correctly classifying the sneaker but the outputs of the network are high for both sandal and sneaker. The confidence isn't so clear cut.

Looking at this confidence is a useful enrichment to understanding the otherwise simple output of a network.

Overall Barney's walk through demonstrated the key stages of machine learning and highlighted some key issues, such as distinct training and test data, and understanding the confidence of a prediction.

Local Apps, AI in the Cloud

Barney then explained a useful architectural approach of having a lighter local app, perhaps a web app running on a smartphone, backed by a machine learning model hosted in the cloud which benefits from larger compute resources.

As a fun example, Barney did a live demo of a smartphone web app took a photo of scone and used a cloud hosted pre-trained model to determine if it was a Cornish or Devon scone!

Conclusion

Barney succeeded in conveying the key steps applicable to most machine learning exercises, whilst also showing how easy modern tools and technology make this process.

Both Barney and David also highlighted that although the tools and algorithms appear impressive and confident, it is important to look beneath the simple outputs to understand the confidence of those answers. David did this with sentiment analysis and Barney illustrated this with image classification.

Quite a few members said they were inspired to try the tools themselves.

Sentiment Analysis - A Hands-On Tutorial With Python

This month we hand a hands-on tutorial taking us through simple sentiment analysis of natural language text.

The slides are at: [PDF]

Code and data are at: [github]

Natural Language and Sentiment Analysis

Natural language is everywhere - from legal documents to tweets, from corporate emails to historic literature, from customer discussions to public inquiry reports. The ability to automatically extract insight from text is a powerful one.

The challenge is that human language is hard to compute with. It was never designed to be consistent, precise and unambiguous - in fact, that is its beauty!

In the broad disciplines of natural language processing and text mining, sentiment analysis stands out as particularly common and useful to many organisations. Sentiment analysis aims to work out whether a piece of text is being positive or negative about the subject of discussion.

This sentiment analysis can result in a simple number, or an even simpler positive / negative label. Even this simplicity can be really useful, providing insights into large or rapidly emerging text, where it would not be feasible to read and assess the text manually.

We were lucky to have Peter give us an overview of sentiment analysis and lead a hands on tutorial using Python's venerable NLTK toolkit.

Two Approaches

Approaches to sentiment analysis roughly fall into two categories:

• Lexical - using prior knowledge about specific words to establish whether a piece of text has positive or negative sentiment.

• Machine Learning - training a model using examples of positive and negative texts. Often that model is probabilistic, that is, it learns the probability of positive or negative sentiment based on the combination of words present in the text.

Peter created two simplified tasks for each of these approaches.

Lexical Approach

A very simple lexical approach is to have a set of words which we know contribute a negative or positive sentiment.

This picture shows just five words.

The word poor indicates a negative sentiment. The word bad indicates a stronger negative sentiment. The word terrible indicates a really negative sentiment. The scores associated with these words reflect how strong that negative sentiment is.

Similarly, the word good suggests a positive sentiment, and the word great suggests stronger positive sentiment. The scores reflect this too.

This is just a very small sample of scored words, but researchers have created longer, more comprehensive, lists of such words. A good example is the VADER project's list of words and their contribution to sentiment: vader_lexicon.txt.

A particularly simple way of using these scored words is to simply add up the scores as we find the words in the text being analysed.

You can see in this very short film review we've found the words poor, terrible and good. Adding up the scores for those gives us a total of -5. The positive sentiment of good wasn't enough to outweigh the very negative sentiment from the first sentence.

This is a simple approach which serves to illustrate the lexical method for sentiment analysis.

You can see that in practice, if we want to compare scores across reviews, we'd need to adjust the scores so that very long sentences or texts aren't unfairly advantaged over shorter ones. A good way to do this is to divide the scores by the number of words in the text snippet. Even this might be improved by dividing by the number of words actually matched and scored, otherwise there is a risk of long passages diluting sentiment scores.

In our simple example, that score would be -5 / 3 = -1.67. The negative result indicating an overall negative sentiment.

A key message that Peter underlines was that there is no perfect method, and each approach, simple or sophisticated, has advantages and weaknesses.

Peter provides a data set of movie reviews, and steam game reviews, and introduced key elements of Python to help us write our own code to calculate sentiment scores for these reviews.

In trying this, some of us found that the review text needed to be lowercased because the VADER lexicon of sentiment scores was lowercase.

The class had great fun trying out this simple example, and it was great to see more experienced members helping those less experienced with Python coding.

Peter's own code where he explores additional ideas like calculating the sentiment sentence by sentence:

• https://github.com/PeterConway314/Jan2019_NLP_Talk/tree/master/examples/Task1%20-%20Sentiment%20Calculation

Machine Learning A Sentiment Classifier

We didn't get time in the session to try the second approach of training a model with examples of positive and negative text.

Peter did discuss a simple approach training a Naive Bayes Classifier.

The Bayes theorem is often difficult to understand when coming across it for the first time, so Peter pointed to an easy explainer on youtube. Essentially, it provides a way of calculating the probability of something given something else has happened (which also has its own probability). For example, what's the probability that it is raining, given my head is wet? You'll hear the term conditional probability to describe this idea.

How is this relevant to our task of sentiment analysis?

Well, we're trying to work out the probability of a piece of text having positive or negative sentiment. That probability depends on the occurrence of words in that piece of text. And each of those words has a likelihood of being in positive or negative texts.

Have a look at this simplified example.

If we look at a training set of negative documents, we might find that the probability of the word poor occurring is 0.8. We'd do this by counting the occurrence of the word.  The word poor might occur in text documents which are assigned a positive sentiment, but they're less likely. In this example they have an occurrence probability of 0.1.

Similarly probabilities for the word good and apple can be established. No surprise that the probability of good in positive texts is 0.7, and a low 0.1 in negative samples.

So how do we use this to help classify a previously unseen document as positive or negative?

Imagine a new previously unseen document only contains the word poor. What's the probability that it is a positive document? What's the probability it is a negative document? Intuitively we know the document is negative, and looking at the numbers the probability of poor being in the negative classification is much larger than the positive.

That's the intuition - and it's not so complicated.

The Bayes theorem just helps us calculate the actual probabilities. Why do we need to calculate them, surely our intuition is enough? Well, that word poor was likely from a negative document, but there's a small chance it could have been from a positive document. That's why we need to take more care over the competing probabilities.

Let's take the key formula and apply it here:

P(negative given poor) = P(poor given negative) * P(negative) / P(poor)

We want to work out the probability of the document being negative given that it has the one word poor. That's the left hand side of the equation. Let's look at the right hand side of the equation:

• The probability of poor given the document is negative. That's what we know from the training data. That's 0.8. 

• The probability of a document being negative. We've assumed a half-half split of positive and negative documents in the training data but this might not be the case. It may be that negative documents are just more likely to occur, just as many reviews tend to be negative because that's when people are motivated to write them. For now let's assume an equal split so this is 0.5. 

• The probability of the word poor itself occurring at all, irrespective of positive or negative document, is something we have to find from the data set itself. If the word is rare this probability is low. In our example, the probability of poor is (0.9 + 0.1)/2 = 0.45.

The means the probability of the document being negative is 0.8 * 0.5 / 0.45 = 0.889.

Doing a similar calculation for the probability of the document being positive if the only word it contained is poor, we get 0.1 * 0.5 / 0.45 = 0.111.

So having a document with the one word poor, the probability that it is a negative sentiment document is far higher than it being a positive sentiment document.

This all looks overly complicated, but we do need this machinery when our training data has an uneven number of positive and negative documents, and when we extend the idea from one word to many.

What we've done is classification. And in fact we can use this very same idea to classify documents against different kinds of categories - spam versus not spam being a common example.

You can see Peter's own code that uses the NLTK Naive Bayes Classifier:

• https://github.com/PeterConway314/Jan2019_NLP_Talk/tree/master/examples/Task2%20-%20Sentiment%20Classification

Care!

What we've looked at are simple ideas that may not perform very well without further preparation and optimisation.

A key reason for this is that natural language is not consistent, precise, and unambiguous. Natural language has constructs like "not bad at all" where considering the individual words might suggest an overall negative sentiment. Sarcasm and humour have been particularly challenging for algorithms to accommodate.

Improvements can include using "not" to negate the sentiment of the subsequent word or few words. Another approach is to consider word pairs, known as bigrams, as pairs of words often encapsulate meaning better than the individual words.

Peter raised the issue of asymmetry in the lexical approach. The strength of "not bad' is not equal but opposite to "bad", and the same for "good" versus "not good".

In terms of assessing and comparing the performance of classifiers, Peter touched on the issue of precision, recall, and the F1 measure that combines them.

Conclusion

Peter succeeded in framing the complex challenge of natural language, introducing two simple methods that represent two different and important approaches in the world of text analysis, and also providing an opportunity for hands-on learning with supportive friends.

Further Reading

• VADER sentiment analysis explained: http://datameetsmedia.com/vader-sentiment-analysis-explained/
• An intro to using VADER for social media sentiment analysis: https://medium.com/analytics-vidhya/simplifying-social-media-sentiment-analysis-using-vader-in-python-f9e6ec6fc52f
• Bayes Theorem, A Visual Introduction for Beginners - https://www.bayestheorem.net/

How To Design For Big Data

This month's meetup was on the topic of designing technology services for growing data.

We also had short talks on making home sensor data open for exploration, and proposing that AI improves game narratives.

A video of the talks is on the group youtube channel: [link]

Slides for Designing For Big Data are here: [link].

Slides for the Smartline project are here: [link].

Slides for the AI Game Narrative talk are here: [link].

Short Talks

Ian Mason from Exeter University explained the Smartline project for-social-good research exploring sensor data from homes in Cornwall, and an invitation to participate in data mining hackathons.

The project is using sensor data installed in selected Cornwall homes, and is investigating how that data can be used to improve the lives of residents, focussing on health and happiness. Examples include predicting house infrastructure faults, and monitoring energy consumption and cost.

The project is looking to open up the data for the wider community of data scientists and businesses for open-ended research to provide additional value. This work could lead to further funded research or the development of business products.

The nexts steps are a formal launch of the data [event link], and subsequent workshops organised by Data Science Cornwall.

John Moore's talk proposed that, although games have drastically improved visual detail, rich interactivity, audio effects and overall immersion, the core narratives and storylines have not improved significantly. He suggested the development of AI techniques to automate the development of such narratives and plot-lines. This is a unique idea, and worth exploring. A suggestion from the audience was to explore existing efforts for generating works of fiction as a similar challenge.

The Data Success Problem

A successful product will often see a growth in the data it stores and processes.

The downside of this success is that:

• data storage can grow beyond the limits of a single traditional database, and 

• data processing can outgrow the processing and memory limits of a single computer.

Rob Harrison is experienced in designing, and remedying, architectures which support larger and growing data.

Scaling Basics

In the past, a common approach to dealing with growing data and processing needs was to grow your computer - faster processor, larger storage and bigger memory - vertical scaling.

This can work up to a point, but very quickly the cost of large computers escalates, and they remain a single point of failure. Ultimately you won't be able to buy a big enough machine to match your data growth.

A better approach is to use multiple but relatively small compute and storage nodes. Collectively the solution can have a larger storage, processing or memory capacity, enough to meet your needs. Benefits of this horizontal scaling approach are:

• cost - each unit is relatively cheap, and collectively cheaper than an equivalent single large machine.
• resilience - if designed correctly, failure of some nodes isn't disastrous, and can often appear to end users as if there was no problem.
• parallelism - lots of compute nodes give the opportunity to processes data in parallel, improving performance for amenable tasks.
• growth - the ability to incrementally add more (and just enough) nodes to meet growing demand.

This fundamental shift from vertical scaling to horizontal scaling has driven a wide range of changes in software and infrastructure architecture over recent decades, from the emergence of multi-core computing to distributed databases.

Assessing Products & Solutions

Rob presented a framework for assessing products and solution architectures. The key points can also be considered design features and principles for designing your own solutions where proportionate to your needs.

He noted that a good design doesn't limit storage - that would not be useful if your data grows. Furthermore, it should scale not just data reads but also writes. The latter is a tougher requirement and some products don't do this well.

Good architectures should be resilient to failure of individual components, and this is typically achieved through redundancy. Related to this is the ability to failover automatically. Too often, products remain off-line for hours as they fail over, which doesn't match modern customer expectations.

Rob made an interesting point about horizontally scaled solutions which don't work if there is any heterogeneity in technology versions. In some sectors, it is a requirement to run several versions of a technology to avoid total failure where a fault affects a specific version.

He also made a point about technology that is aware of its physical location, within a rack or a geographical zone, to better optimise data flows amongst nodes.

An important point made by Rob is that any solution shouldn't lock you into a single vendor's products, or indeed a single cloud. This requirement for sovereignty over your own data underlines the importance of open source technology and cloud vendor agnostic platforms, such as Docker.

State Then And Now

State is just a word that describes the information in, and configuration of, a system. Applied to technology products, the state often refers to user data. It is this data that grows as a service grows.

The complexity of state is the major driver of how complex a system is to manage and scale.

The standard architecture of many technology platforms is often described as in this diagram.

It shows a user device accessing a service over the internet, with requests being distributed to one of several web servers by a load balancer. We can add more web servers to meet demand if needed. So far that's fairly resilient and scalable.

The diagram shows all those web servers sharing a common database and perhaps a common file server. These are the single points of failure and potential bottlenecks.

Rob then shared a more complex architecture that more truthfully reflects the reality of today's services.

Often the application logic is not located in web/application servers, but in an app on the user's device. That device will have its own local data store. The device will connect to API servers over the internet, but that connection is rarely permanent, and sometimes poor. The server side datastore is often not a traditional relational database but a less-rigid data store that can store data structures that better match the application, JSON documents for example. The term NoSQL has emerged to describe a broad range of such data stores.

Many of these modern NoSQL data stores offer a range of choices for how they scale, allowing developers a reduction in consistency in return for much easier scaling and resilience. Many applications don't need, or can tolerate, a write operation taking its time to asynchronously replicate to multiple nodes.

Rob's picture also refers to OS for object store. These store data in the form used by applications, and not broken into fields and forced into tabular form, only to be reconstituted when queried. He also refers to PNS for publish and subscribe - a pattern for publishing messages to be picked up by interested subscribers when they're ready. This loosely coupled approach works well for synchronising state across intermittent internet connections.

Another advantage of some modern NoSQL data stores is their flexibility with data schemas. That is, they don't firmly insist that stored data all conforms to the exact same schema. This makes much easier the iterative agile development of applications. An up-front one-shot data design is unlikely to meet future needs as a product evolves.

Rob offered some wisdom in his analysis of the emergence of modern NoSQL data stores:

• Traditional relational databases emerged from a time when storage was expensive, and so huge effort was put into normalising data to reduce duplication. Today storage is cheap, and the cost of normalising and denormalising, and the risks of a fixed schema, are no longer tolerable. 

• Today's data stores are better matched to the needs of application developers. They are able to store objects in formats closer to those in the application, such as key-value pairs and JSON objects, binary objects and even graphs of linked entities. Flexible schemas support the reality of applications that iterate and evolve.

• Today's data stores aim to meet internet-scale demand and the availability expectations of empowered customers. 

What's So Wrong With Traditional Databases?

Rob discussed the challenges of traditional databases throughout his talk so it is worth summarising the key points:

• Relational Database Management Systems (RDMBS) were, and still are, the most common kind of database in use. Today's popular databases like PostgreSQL and MySQL, though extended with modern capabilities, have their roots in decades old design assumptions. 

• One assumption is that data storage is expensive, and that effort put into normalising data, linked by keys, is worth the effort of decomposition and reconstitution. This assumption isn't particularly valid today.

• Although the data model of fields in tables linked by keys can be useful, it can sometimes lead to catastrophic performance failures when the right indexing isn't anticipated or the joining of such fields becomes complex.

• Many traditional relational databases aim to be ACID compliant, which can be simplified as being always correct at the expense of latency. This can lead to performance bottlenecks as an entire table or even database is locked during an update. Many of today's applications don't need this level of transactional consistency, and many only require eventual consistency. 

• Traditional relational databases weren't initially designed to be horizontally scaled across nodes. This means that attempts to make them work this way are more of a retro-fit than an engineered solution. Issues and challenges include locking across nodes and inability to handle data inconsistencies caused by inevitable network failure, exacerbated when the links are across distant geographies. Aside from infrastructure, the challenges continue at the logical level, for example, keeping universally unique identifiers for database keys consistent is a challenge across multiple instances of a traditional database, particularly when rebuilding a failed database. 

A standard approach to scaling relational databases is sharding - splitting your queries amongst your nodes. The following picture shows the simple splitting of database queries so those related to user names beginning with A-D are processed by the first shard, and those with U-Z are processed by the last shard.

Although this seems like a fine idea, it fails when a shard can no longer meet demand and further scaling is needed. Unique database keys make migrating data to a new architecture very difficult, without rebuilding all the data again. Another problem with sharding is that the demand profile can change leading to over-used and under-used shards. Again, rebalancing the queries requires a database rebuild as we can't simply shift data from one shard to another.

In summary, traditional relational databases were fine for the time they were initially designed. They are useful today, but we now have more choices for data storage and processing that better match modern internet-scale design goals and user expectations.

Modern Tech Stack

Rob the talked us through the technology stack from low-level memory up to filer servers, comparing historic approaches and modern technologies.

Memory Is Faster Than Disk

An early method of accelerating traditional web applications was the use of memory-based caches. That is, temporary in-memory stores that very rapidly served responses to queries that had been seen before, and for which the response had been calculated one before.

Although the original setting for these caches was web queries and responses, the idea easily generalises to other kinds of queries and responses, including data store queries and responses.

Today these memory-based caches can coordinate across multiple nodes, allowing the scaling of the cache beyond the limits of a single machine.

Memcached and Redis are to popular choices for a memory-based cache for data stores, with memcached being a better choice for simpler and smaller data structures, and redis being better for more sophisticated data structures.

Such memory-based caches should be considered as accelerators reducing the load on data stores, and not as key methods for scaling the stores themselves.

NoSQL

Rob proceeded to discuss examples of so-called NoSQL databases, built to different design goals from traditional relational databases, and which aim to be much more easily scalable and flexible for application developers.

To explain the terminology, SQL is the query language most commonly used with relational databases. It was developed in the early 1970s! Modern data stores initially didn't use this language, so they became termed No-SQL, but today some do offer the ability to query them using SQL-like queries and so No-SQL can mean "not-only" SQL.

Rob explained how Google was an early leader with BigTable which can be thought of as a two-dimensional key-value store, with one key being a row and the other being a column identifier. The contents of the value don't need to conform to a schema. Google describes their public BigTable service as suitable for petabyte scale data whilst still providing a performant services (sub 10ms latency). A number of Google's own services are thought to run over their own BigTable implementation.

Today BigTable supports a wide range of businesses, processing a wide range of data from financial time series data to user profile data.

Facebook open-sourced its implementation of a similar wide-column database, Apache Cassandra. Because it is open source, we can see how it works. Key design features include:

• Distributed across nodes, where every node has the same role, if not data. There is no single-point of failure, and no concept of a master or slave hierarchy. 

• Aim to scale both read and write fairly linearly as nodes are added.

• Fault-tolerant to node failure, and designed for nodes to be replaced with no service downtime.

• Tuneable consistency, from "issue write and don't worry when it completes" to "block everything until data written to at least 3 nodes, and they all confirm it".

These are typical objectives of several modern NoSQL data stores, and the contrast with traditional databases is stark!

Cassandra has been used successfully by organisations such as CERN, Apple, and Netflix.

Rob also mentioned Couchbase, which is often used to store JSON objects, ideal for application developers and supporting REST APIs. The following chart shows its write performance compared to other NoSQL databases. Couchbase remains performant at 20,000 writes per second compared to MongoDB which degrades at 6,000 on comparable systems.

You can read more about how Couchbase on Google Cloud infrastructure reached 1 million writes per second, of over 3 billion items, using just 50 nodes [link].

File / Object Storage

Sometimes our data just isn't particularly structured and would normally be stored on a file server.

Modern options for storing arbitrary files or objects include the popular Amazon S3 service. Although not particularly fast, such services are very cheap and very resilient, offering availability of 99.9%, which means it should not be down for more than 43 minutes per month.

After Amazon's lead, other providers have offered competing object storage. Almost all of them are designed to be accessed programmatically, offering easy to use REST APIs. Some even offer APIs compatible with Amazon's S3 to ease development, migration and multi-vendor or hybrid-cloud architectures.

The term object here just means an arbitrary bunch of data, like a file, rather than a structured object used by application code.

An illustrative use for such low-cost, if not very fast, object stores is for user generated photos.

Distributed Computation

Distributing the storage and retrieval of data from one node to many nodes can improve performance and resilience. The same can also work with computation.

Instead of a single node performing calculations on data, the computation can be spread over many nodes. Many, but not all, tasks are amenable to being split into smaller parallel tasks. The performance benefit of performing these tasks in parallel is very attractive.

A good illustration of the idea is the task of counting the number of words in a book. One node can count the words from the first to last page. Alternatively, many nodes could be working on a chapter each, working in parallel. If those nodes weren't sufficiently performant, those chapters could be further divided into paragraphs distributed to further nodes to count.

The most famous example of this approach is Google's MapReduce, an idea which has been implemented by others. Apache's Mahout is a notable example of numerical and machine learning algorithms implemented over a map-reduce framework.

Rob's discussion led to Hadoop, probably the most recognisable name amongst the new wave of data storage and compute technologies. Hadoop is not one, but a collection of technologies including a distributed filesystem HDFS, a wide-column store HBase, a MapReduce engine, and task managers.

Hadoop is often lauded as the solution to many data problems, but its main focus is distributed computation, and other technologies may be better choices if your task is storing and retrieving data but not distributed computation.

Functional Programming

Rob also shared his thoughts on application development. One theme was particularly interesting.

It is a reality that many programming languages don't protect against accidental changing of data beyond the intended scope. As application code and logic gets bigger and more complex, the risks of these unintended side-effects grows. Add to this mix, parallelism and we have a new class of potential errors.

Functional programming languages impose a restriction on functions such that they can't have any side-effects, except those that are explicitly intended, and even these are tightly controlled. This is done not by complexity but simplicity. The benefit is that sophisticated and complex applications can be composed of these basic functions, and we can verify that the fuller code also doesn't have unintended side-effects.

As a bonus, the implementation of these functions requires them to independent in operation and that makes them very easily parallelisable. Functional programs naturally benefit from multi-core and multi-node hardware without additional effort on the part of the developer.

The last 5 years has seen a growth in demand for functional programmers in some sectors, because large complex distributed applications are easier and safer in functional languages.

Conclusion

Rob's talk was well anticipated and well received. Many members commented that his talk had forced them to rethink their assumptions and designs, or broadened their options for future projects.

My own summary of Rob's message is that today's data technologies

• aim to support internet-scale services, and the high expectations of modern users.

• are designed from the ground-up to scale horizontally.

• allow application developers to choose their own balance between performance, consistency and availability. 

More Reading

• Wikipedia provides a good overview of NoSql data store types https://en.wikipedia.org/wiki/NoSQL
• A very good introduction to the why and how of NoSQL https://www.amazon.co.uk/dp/0321826620
• A plain English intro to the CAP theorem: http://ksat.me/a-plain-english-introduction-to-cap-theorem/