Model Diagnostics Addin

Now available on the JMP File Exchange.

front-summary

I’m super-excited about this.  I have been reworking an old add-in that I use to generate model diagnostic information.

The add-in is designed to make it easier to conduct residual analysis in order to check model adequacy and to investigate possible missing terms.  These diagnostics have been enhanced but what I’m really excited about is that it’s now possible to run diagnostics not only on open model windows but also from table scripts.  So if you have a table open, containing table scripts generated from the Fit Model platform, the add-in will automatically identify those model scripts and generate model summary information as well as detailed diagnostics.  In the case where you have models relating to multiple responses, the models are listed per-response.

Here’s an example of a workflow.

I open the sample data “Tablet Production” and run the add-in.  The add-in shows me that there are 3 least square models relating to the response Disso (tablet dissolution):

disso

The 3rd model has the best performance statistics.

In order to illustrate the use of the diagnostics, let me choose the first model, which is less than optimal.  When I click on the model name I can drill-down and see further information about the model:

model-terms

The effects tab suggests that there may be some higher-order active effects:

effects-diag

  •     A*A   =   quadratic effect of mill time
  •     G*F   =   interaction between spray rate and coating viscosity
  •     C*G   =   interaction between blend time and spray rate

The above effects involve 4 factors.  The residuals tab allows me to explore my residuals in relation to all the table column variables.  The residuals plotted against mill time show distinct curvature.

milltime-resid

The Partition platform is also used to partition the residuals:

resid-part

More evidence of the need to include higher-order terms for mill time.

Based on this information I could launch the table script for this model and make revisions to the model:

revising-model

Once the model edits are committed I remove the terms that are not statistically significant:

effect-summary

Now I want to perform diagnostics on this new model.  That’s easy.  I can refresh the contents of teh model diagnostics addin.  When model windows are open, these models are summarised instead of the models contained in the table scripts.

So I can see my original model, and the model containing the revisions:

new-models

My new model doesn’t show any patterns in the residuals and a normal quantile plot indicates that they are normally distributed.

normquant

Job done.

4 thoughts on “Model Diagnostics Addin”

  1. David – this is awesome. How would you handle when one model excludes data points that the other model includes? That information isn’t really stored within a saved script for something like the Fit Least Squares platform. I’m sure there’s a method involving extra data table columns with row states but I’d really like to avoid that.

    1. Mary, that’s a good point, the scripts only reference the columns by name – rows could be excluded or even new rows added and we wouldn’t know. Row states would be one way of trying to persist the state at the time the model was created but I think it’s clumsy. When I write scripts that fit models I always grab the table script for the table, so that I can use that script to regenerate the table in the same state as it was when the model was created.

      1. Appreciate the response! I’ll try out your table script method and see if that works for my needs. If you come up with any more thoughts, I’d love to hear them because this is something I deal with constantly.

Leave a Reply

Your email address will not be published. Required fields are marked *