Suppose you just obtained a fresh new set of data. You might know what it’s generally about, but the only way you can know what hypotheses to test (and what analysis challenges you’re up against) is to explore the data and audit its overall quality. Unless you can see the basic overview of what you’re dealing with, you won’t be able to determine the most appropriate models to build or develop a strategy for handling challenges such as outliers and missing values.

The constant stream of breaking news, real-time updates, and individual opinions about events, products, or people is a rich source of information that can turn into valuable insights and business intelligence —if extracted in the right way.

Twitter offers an API platform that allows the search and collection of tweets from the public timeline using keywords or the feed of specific accounts. Currently, there are three different tiers of search APIs: Standard (free), Premium, and Enterprise. These tiers differ with regard to data limits, search operators, and technical support from Twitter. In most cases, creating a developer account to use the Standard API is sufficient.

But how exactly does the Twitter search API work?

Let’s say that we would like to collect the latest tweets that talk about tasty food. The query may be written as:

• tasty food, returning tweets containing both tasty and food, but not necessarily in this order
• “tasty food”, returning tweets with the exact phrase tasty food
• tasty OR food, returning tweets containing either tasty or food
• #tastyfood, returning tweets containing the hashtag #tastyfood

Of course, there are more operators listed in the Twitter API documentation for narrowing our search, some of which are only available for the Premium and Enterprise accounts (premium operators). The Twitter API also has the option of returning the latest tweets, the most popular tweets, or a mix of the latest and most popular tweets; in this case, we have used the option for the latest tweets.

Once a query is submitted, the API sends a request to retrieve the tweets that match the query. Depending on the account tier, the API has limits on the number of requests, as well as different windows of time before exceeding those request limits. The most commonly used Standard API allows 180 requests per 15-minute window that will go back 7 days in the Twitter archive. Each request returns 100 tweets, so we can retrieve 18,000 tweets every 15 minutes until we reach 7 days back in the archive. If our task required access to tweets in the last 30 days—or even the full Twitter archive that reaches back to 2006—as well as a greater number of retrieved tweets per request, then we would need a paid Premium or Enterprise API account.

The data is returned in JSON format and includes the actual tweet along with metadata about each tweet and the user who created it. Consequently, a JSON parser is required to have all this information “translated” into a tabular format and/or imported into a database.

The time it takes to collect our data depends on the number of tweets matching our query, as well as the request and time-window specifications of our tier. To continue with our previous example: if we have a Standard API account and our query for #tastyfood matches 500 tweets within the 7 past days, it will take 5 requests and a few seconds to collect the tweets. But if there are 90,000 tweets within the past 7 days that match our query, it will take 900 requests and five 15-minute windows (a little over an hour) for our data collection.

Even though the free Standard search API only allows us to go back 7 days, we can still collect longitudinal datasets by moving forward instead of backward. For example, we can automate the task of sampling tweets every 1 hour each week. Keep in mind, if it is a highly active topic, we may lose some tweets between our sampling windows; and if it is a less active topic, we may have some repetition in our dataset that we would need to clear out.

One of the main barriers for researchers and companies who need to collect data from Twitter is the requirement of basic programming skills to use the API and its output. For this reason, Megaputer has created a user-friendly Twitter data collection feature in its software, PolyAnalyst™, which takes the programming part out of collecting the data by using a convenient graphical user interface (GUI). Users only need to input their account information, their query, and select further search options offered by the Twitter API (i.e., the language used in the tweets). PolyAnalyst™ then collects the data, automatically parses the returned JSON output, and imports the data into a table for further analysis. In addition, the built-in Scheduler feature allows users to schedule longitudinal data collection over a specific period of time, which will then update the data analysis workflow to include the most recently collected data.

Try it out yourself by requesting a free trial.

Neural Whatnow?

Neural Networks. You’ve probably heard of them. They are at the forefront of the machine learning craze and are the driver of many of the most impressive advancements. The technology that led machines to the best humans in Go and the popular video game Starcraft? Neural Networks. The backbone of algorithms that can recognize images and faces, which are igniting a surveillance and privacy panic? Neural Networks.

And yet, Neural Networks aren’t some new idea spawned from the incubator of a giant tech company. They aren’t some stroke of genius from a college student turned dropout who went on to found a revolutionary tech firm. Neural Networks are in fact… old hat. Or they were.

The concept of a Neural Network (something we will get to later) has been around for years. They date back to the 1970s, and simpler versions of them existed even in the 1940s! So, if they have existed for decades, why are they only popular now?

The answer is related to the hardware and data advancements mentioned earlier. Neural Networks crunch a lot of numbers. They also need a lot of data to help them learn. Up until the past decade, this made training anything but the simplest networks highly time-consuming and expensive.

With all the great improvements to hardware over the years, the possibility of using more advanced Neural Networks became possible. Aided by hardware demands from the entertainment industry and now cryptominers, GPUs (graphical processing units) have been developed which can calculate specific mathematical operations at lightning speed. Luckily, these same kinds of operations occur in training neural networks. Using the technology originally developed for beautiful visuals in film and video games helps us train networks at a fraction of the time it takes a traditional CPU.

What’s in a Name?

The power of Neural Networks may be evident, but at this point, one may also be wondering what they are in the first place. The name gives some clues. A Neural Network isn’t a singular object that makes decisions by itself. It is, as implied, a network of smaller objects all connected. A network of what, then? We call them Neurons. Neurons as in the cells inside our brains? Yes! Well, no. But sort of!

Neurons in Neural Networks can be thought of as being like the neurons in brains. They are tiny, individual units that are connected to other neurons in a large, structured network. These connections allow tiny pieces of data to flow between them. In our brains, these are electrical pulses. In the Neural Network, we send numbers between Neurons. The Neurons then take all the numbers fed to them by the Neurons they are connected to and process them. The process isn’t complicated—in fact, it is painfully trivial. After all, it is just a tiny unit—a single cell in our brain. But it then sends the processed information out to other Neurons it is connected to. Another number. Another electrical pulse. And so on. Tiny Neurons are fed tiny pieces of information, perform tiny pieces of computation on that snippet, and feed it forward to other Neurons, which do the same over and over until we eventually reach a final set of Neurons—the output of which is our final result.

From the collective effort of many small individual units networked together in a perfectly calibrated balance, we can achieve enormous computational power. The Whole is greater than the Sum of its parts.

 

Balancing Act

We are pleased to announce that PolyAnalyst now supports modeling data using convolutional neural networks. A convolutional neural network, also known as a ConvNet or CNN, is a particular kind of neural network. Understanding what a ConvNet does is simply a matter of understanding the general concept of a neural network, and then focusing on the distinct characteristics of the convolutional adjective.

Neural networks are one of the most popular methods for solving machine learning problems such as:

• Classification – assigning data points to a hierarchy of classes, also known as labels. For example, determining who you might vote for based on where you live and your income, and then labeling you as a particular kind of voter.
• Regression – measuring how a numerical value changes based on the values of other inputs, typically for predicting a future value. For example, determining how likely it is to rain tomorrow, or how likely you are to be a repeat customer.
• Clustering – measuring the similarity of data points and attempting to group them. For example, identifying typical consumer markets based on purchasing behavior.

Convolutional networks are not that different from conventional neural networks. What sets ConvNets apart is their ability to process signal data, such as the pixels of an image, with remarkable efficiency. ConvNets are particularly well suited to finding patterns in noisy data, and tend to be more scalable than other neural network algorithms.

A brief history

Neural networks are certainly not new. A quick search will yield academic papers dating back to the 1940s. However, the convolutional flavor is newer, and has in recent years surged in popularity due to a renewed focus on deep learning.

Historically, ConvNets were used to process image data in an academic field known as computer vision¹. ConvNets are notable for how well they lend themselves to the task of recognizing features of images, such as people’s faces, or the continually-updated external view of the surrounding environment in a self-driving car. ConvNets are also useful when processing natural language (unstructured data), and for optical character recognition (OCR).

Applications of convolutional networks

• Facebook uses convolutional neural networks to assist in automatically choosing how to tag Facebook posts.
• Google uses convolutional neural networks to assist in providing accurate search results for Google Images.
• Amazon uses convolutional neural networks to assist in recommending products to customers (cross-selling).
• Pinterest uses convolutional neural networks to assist in personalizing your home feed.

What are the advantages of convolutional neural networks?

CNNs consist of convolutional layers, which we will touch on in more detail in a moment. CNNs allow you to extract features and build a multi-layered hierarchical structure from them. In this structure, features at a higher level are derived from features at a lower level.

Convolutional layers have a smaller number of weights than fully connected layers. This increases the efficiency of training multilayer networks.

While this approach is suitable for pattern recognition tasks, it can also be applied to numerical data.

What is convolution?

Convolutional neural networks are so named because of a data processing step known as convolution. A thorough mathematical treatise on what is convolution is outside the scope of this brief article. It is sufficient to emphasize that convolution is good at extracting features from data, and more so in its aggregation and filtering of those features, so that the number of features is reduced (a.k.a. pooling). This built in feature reduction is what makes such networks so adept at analyzing feature-rich data.

What is feature reduction?

Let’s take a step back from the technical jargon. We are going to aim for intuition in our explanation.

Feature reduction is like drawing a sketch. Imagine laying a thin piece of paper over an image, so that in a certain light, only some of the features of the image bleed through the paper. Take a pen and outline some of the lines. Then, discard the original image, and examine your sketch. The drawing contains substantially less information about the image. The original color is gone. Only some lines remain. In academia, this might be known as the application of an edge detection algorithm. Each line you have drawn is a recognized edge. As easy as it was for you, getting a computer to do this is not the easiest feat.

One of the notable characteristics of convolutional neural networks is that this edge detection step can be baked into the modeling step itself. In other approaches to machine learning, the steps of preparing the data for analysis are typically separated from the learning step. Here, a part or all of the cleaning step is merged with the learning step.

The first layers of the network serve to abstract away some of the features of the input data, and the later layers then continue processing these abstracted features as a substitute for the original. Because real world data may have thousands of useful features that could serve as decision-making criteria, it would ordinarily be cost-prohibitive to efficiently model the original data. This step of reducing the number of features via convolution and pooling in a way that increases modeling efficiency is why convolutional neural networks are so attractive.

We should point out again that this is a simplification. Real networks may involve multiple layers of feature extraction, and multiple layers of pooling. Feature refinement is an iterative process. Each time the network examines the data in its training phase, there is an opportunity to make adjustments.

Juggling the criteria for decision-making

There are trade-offs in reducing features. If the algorithm excises too many of the original features, the result is decreased fidelity.

If you recall the childhood game of Telephone, where children sit in a circle and starting with one child pass a whispered message from ear to ear, the final message is often hilariously different than the original. Each repeated utterance of the original message involves a loss of information. At some point the loss accumulates and the message changes.

One of the benefits of coupling the feature extraction step with the later layers is that the network can reconfigure itself as it memorizes new patterns. For example, if the network is doing a bad job of predicting a value, as measured by a high error rate, then the network can perform additional training that starts to consider additional features, or consider the same features in greater detail, or stop considering certain features, or view things from a different angle.

For solving the task of feature extraction, the addition of more data points to consider is not always a good thing. As more features get introduced, this can greatly increase learning time, and sometimes lead to a loss in accuracy due to an overabundance of features, similar to the difficulty of picking a person out of a crowd as the crowd size increases.

Step back and think about how good your ears are at filtering out noise from a crowded room of people to listen to one person speaking, or how adept your eyes are at filtering out information from your peripheral vision. The problem for machine learning is that machines see and hear all data indiscriminately. Machines do not come with this innate human ability to ignore objects in your visual periphery, or tune out annoying sounds and static. Machines struggle to distinguish between signal and noise.

This balancing of what data to consider and what to ignore is a juggling act. It requires some finesse. Perhaps even some subjectivity. Convolutional neural networks offer several dials to turn to achieve that. The networks mentioned in the earlier examples are highly refined.

Learning from mistakes: how neural networks evolve to make better predictions

Neural network algorithms are especially susceptible to a modeling problem of settling on a false minimum in the error calculation.

Imagine climbing halfway up Mt. Everest and then claiming you reached the summit simply because you could not see the top of the mountain from your vantage point.

Neural networks learn by minimizing error. This is similar to classical behavioral conditioning, like Pavlov’s dogs responding to the sound of a bell. A network will produce a guess, a prediction, of what a value should be, then measure the error, and then get rewarded or punished. If it gets zapped for picking a bad value, it measures how far off it was from the correct value, and then rewires its neurons so as to produce a guess that is hopefully closer to the desired value. Enough zaps, and its answer becomes more accurate (assuming that the data at least exhibits some semblance of a pattern).

However, networks are prone to settling on what a network thinks is the most accurate guess when in fact it is not. The algorithm cannot know in any given training iteration that it is finally accounted for all decision-making criteria. Instead, if the algorithm sees that after several attempts to tweak its criteria it fails to produce appreciable impact on reducing error, it gives up. It is often the case it has only climbed part of the way up or down the mountain.

The point is that no algorithm is a silver bullet. There is no one-size-fits-all machine learning approach. Be open to experimenting with other modeling approaches such as decision trees or regression.

For the most part, the other steps of any ConvNet algorithm are highly similar to a traditional neural network, so we will not be going into much more detail.

PolyAnalyst’s new Convolutional Neural Network node

PolyAnalyst recently introduced dedicated functionality for training a convolutional neural network model. The new node provides several options to configure. While many software packages that provide similar functionality tend to require an analyst to explicitly configure the network’s structure, PolyAnalyst’s implementation enables automatic structure building. In particular, PolyAnalyst’s implementation can filter and normalize the data, form a validation sample to control overfitting, train the network with the selected architecture, split the sample into training and testing batches, and ultimately select the best model.

An example of using the Convolutional Neural Network within PolyAnalyst

Speaking to readers familiar with PolyAnalyst, you can use the new node just like you would use any of PolyAnalyst’s many other modeling nodes such as classification and regression.

It is important to prepare the data prior to training. Once the data is in a suitable form, you will need to specify the dependent variable and the independent variables. In the following data analysis scenario, we see the input data split into a training subset and a testing subset. The model is trained on the training subset, and then tested against the testing subset.

The following screenshot communicates a portion of the report that enables you to assess how well the model performed. Here, you can see how the model learned patterns in the data in an iterative fashion. This particular chart shows how the prediction error, that is what the model thinks the predicted value should be compared to what it actually is, decreases over time (with increasing number of training epochs) as the model observes more patterns and performs retraining procedures.

This error chart can be useful for adjusting training parameters in future runs of the model.

The presence of such a solution also allows:

• supporting different architectures and types of neural networks,
• processing new types of data such as rasterized images, and
• better processing of data representing a series of events.

If you are interested in learning more about PolyAnalyst, or viewing a free, personalized demonstration of the convolutional neural network, please contact us.

Data analyst Chris Farris takes us through the idea behind machine learning and how this segment of AI is affecting business and our personal lives. Hear his insights for the future of machine learning that will impact life as we know it for the next 50 years.

Brian Howard, Sales & Marketing Manager, talks about the differences between text and data analytics and how we use these insights for better business decisions.

The following is a video of Megaputer’s own Dr. Michael Kiselev talking about pulse neural networks and microchips. This video is only available in Russian.

A brief experience of Megaputer’s PolyAnalyst software. Watch a simple workflow on demonstration data going from data loading to visualization using a few of the many ‘nodes’ accessible for data and text analysis. The video is about 10 minutes in length.