The Power of Journals

If you have ever been to a JMP Discovery Summit or perhaps a JMP user group meeting you will no doubt have come across journal files.  The first time you create one can be an unnerving experience since a new journal is simply a blank window.  But that is their beauty: they are a blank canvass onto which you can save your JMP output.  But more importantly they are a place to capture your thoughts.

I’m often asked what is the main strength of JMP.  For me the answer is easy.  It is a tool that allows me to perform train-of-thought analysis.  That thought process will involve three main activities: data transformation, visualisation and modelling.  But there is two potential pitfalls with this type of analysis; firstly it becomes repetitive – there are certain tasks that are performed over and over albeit as part of different “workflows”.  And second, I am unable to sustain a train of thought for more than a few hours.  Often my work takes days or weeks.  If they are my own personal pet-projects they can take months.  So whilst journals are great for communicating information to colleagues, for me the real power of journals is their ability to keep a thought-process alive even with all of the day-to-day distractions that we have to deal with.

Last month I started a project that involves modelling images of hand-written digits.  I’ve written a few blogs about it, all with the tag hieroglyphics.  SInce then I’ve been busy finishing off a JSL project, I’ve broken my finger, I’ve been robbed, I’ve had a vacation.   Now I want to resume the project: where do I start? I have a project folder, it contains 3 data files, 27 JMP tables, 22 JSL scripts, and 1 journal file.

My journal has a number of distinct sections.

Source Data

A description of the original source data.

Data Tables

This not only gives me access to the data but also provides descriptions – essential for me because I tend to create lots of subsets of data.

Scripts

A better name would be scriptlets.  Whilst there is no restriction on the length or complexity of these scripts, they tend to be very short snippets of code that perform a distinct task.  Individually they don’t do much but by putting a sequence of them together you can construct a workflow.

Often I want to look at the code and maybe revise it for specific tasks so the journal contains a number of ‘view’ buttons that open the script in an editor rather than execute the code.

Workflows

A sequence of steps to perform specific outcomes.  Typically these steps are supported by the above scripts (in essence the scripts section of a library of tasks that I orchestrate in the workflow section).

Here is the journal:

hier-jourSo with all the distractions of the last month I feel confident I can replay my previous trains of thought and start developing new ones.

17 May 2020 Update – I have put together an online course that takes a closer look at the process of building journal files.  If you would be interested in this please DM me on my twitter handle: @PegaAnalytics.

Leave a Reply

Your email address will not be published. Required fields are marked *