All posts by David Burnham

Text Processing Workflow

In my previous post I introduced the sample data table Pet Survey.  I created a column formula to classify each respondent to determine whether they owned a cat, a dog, or both.  In this simple example, there were signs of the problems that arise when processing unstructured text data.  My classification of “dog” missed out responses referring to huskies; my classification of “cat” incorrectly included references to cattle.  I looked at the Text Explorer platform and focused on the output contained in the lists of terms and phrases.  In this post I want to focus on workflow: using the functionality within Text Explorer platform to gain meaningful insights into my data, and to answer specific questions.

(more…)

Segmented Regression

I’m sure there is a more technically correct term for this: I use the phrase segmented regression to describe the process whereby I select a segment of data within a curve and build a regression model for just that segment.

click on the image to see an animated view
click on the image to see an animated view

I have some code to aid the process.  The code illustrates how to perform regression on-the-fly as well as how to utilise the MouseTrap function to handle mouse movement events.

(more…)