University of New Hampshire JMP for Statistics Questions

Paper Helicopter Group Project for Math 740/840I Introduction
The group project consists of designing and running an experiment or
experiments to find the optimum configuration for a paper helicopter. The
basic helicopter design is described in section III of this document and
students must follow the basic design specified in section II. The
experimental factors to be used are also described in section II. Be sure to
carefully read the entire document. Section II contains important
information on the design of the helicopter and the conduct of the
experiment.
Two responses are to be recorded for the experiment. The first and
primary response is flight time, which is how long it takes the helicopter to
land after being dropped from a specified height. The second response is
accuracy, which is the distance the helicopter lands from a specified target
point in the landing zone. An inexpensive plumb bob can be used to align
the helicopter drop point (above) with the target location (below). The same
distance, drop point, and target should be used for the entire project. Your
goal in the experiment is to find an optimum helicopter configuration that
gives the longest flight time and the most accuracy (closeness to target). In
addition to flight time and accuracy consider the qualitative factor stability,
it is highly desirable to have a helicopter configuration with a stable flight
trajectory rather than one that tumbles and falls to the ground. Finally, in
considering flight time, I wish to compare results from different projects, so
convert the observed flight time to a rate of descent in centimeters per
second based upon your drop height – I assume that students will most
likely use different heights. Lower rates (longer flight times) are
desirable.
You may complete the project as an individual; however I do encourage
students to work in groups if possible. If you decide to work as a group,
then I fully expect everyone in the group to contribute to the conduct,
analysis and write-up of the experiment. The same grade is given to each
member of a group.
Finally, try having some fun with the project. It is a nice exercise to
apply the concepts learned in the class and teach design of experiments, but
1
it also can be a fun exercise.
II The Paper Helicopter Experiment
Introduction
The “paper helicopter experiment” is a training exercise, which allows
participants to apply the concepts of Experimental Design to an actual
physical experiment. The exercise involves the design and construction of
paper helicopters with the goal of finding a helicopter design that maximizes
total flight time (the time required for a paper helicopter to land after being
dropped from a predetermined height) and accuracy (the closeness of the
landing point to a specified target in the landing zone). Figure 1 depicts the
general shape and experimental design factors that could be considered for
the construction of a paper helicopter. The three factors you are to
experiment with in this project are bolded in blue in Figure 1. The flight time
response should be converted to a rate so that it is possible to compare
results of helicopters dropped from different heights by the various groups.
As a rule the minimum drop height should be > 10 feet (or > 3 meters) in
order to give the helicopter time to stabilize and rotate once it is dropped.
Experimental Factors
Figure 1 in section III shows a picture of a typical paper helicopter.
Figure 2 in section III shows a schematic of the paper helicopter. Everyone
must use a 23 full factorial design. The 3 factors are Rotor Length, Rotor
Width and Nose Length. All other factors are to be held constant (see the
notes below for the complete list of possible factors). It is up to each group
to determine the high and low settings for each of these three factors. You
may wish to build a few trial helicopters to help decide on the high and low
levels. However these trial helicopters should not be used further and are not
to be included in the data from the actual experiment. Keep in mind when
choosing the high and low settings for each of the three factors that every
helicopter design must fit within an 8.5” by 11” sheet of paper. The
following are a list of all of the experimental factors that exist for the paper
helicopter; see Figure 1. Remember you are to vary only the three factors
specified above; all others are to be held constant.
2
1. Paper Weight – Two types of paper are usually considered. As an
example standard copier paper (20 lb.) and heavier paper (e.g., 40-50
lb.). Hold constant at a single weight of the groups choice.
2. Paper Clips (ballast) – The number of paper clips attached to the
helicopter nose as stabilizing ballast. Hold constant at 1 paper clip.
3. Nose Length – The length of the helicopter nose. The group must
choose a high and a low setting for this factor.
4. Nose Width – The nose width of the helicopter. Hold constant at some
size determined by the group.
5. Rotor Length – The length of the helicopter rotors. The group must
choose a high and a low setting for this factor.
6. Rotor Width – The width of the helicopter rotors. The group must
choose a high and a low setting for this factor.
7. Block Length – The length of the block (in the long direction of the
paper). Note that the rotor width defines the block width as well. Hold
constant at some size determined by the group.
8. Block Bevel – The block is either beveled or left square (90 ). Hold
constant at square (no bevel).
9. Rotor Bevel – The rotors are either beveled or left square. Hold
constant at no bevel.
10. Rotor Struts (supports) – One can add struts to support the rotors by
cutting slits in the block/rotor. See the diagram. Hold constant at no
struts.
Designing the Experiment
Once you have determined the high and low settings for each of the
three factors, and then use the Full Factorial platform in the JMP DOE
3
menu to design the experiment (DOE →Classical →Full Factorial Design).
In the Factor
Table be certain to enter your predetermined high and low settings for each
factor, do not use the generic -1, +1 settings provide by JMP. Do not
replicate the entire experiment; however add three center points (runs) to
the design before making the design table. Please note that for each center
point run you need to construct a new helicopter. In order to capture
experimental error three separate helicopters (EUs) need to be constructed.
Your final design table should have 11 total runs in random order and each
run requires the construction of an individual helicopter – do not reuse
helicopters from other runs.
Prior to the actual experimentation phase of the project, spend some time
brainstorming various unwanted sources of variation that might mask
significant experimental effects. Discuss strategies in the execution of the
experimentation that could remove or at least control these extraneous
sources of variation. An example of unwanted variation might be
differences in the use of a stopwatch for different members of the team or
changes in air pressure or temperature in the room might be a couple others.
In your write up explicitly state the identified sources of nuisance
variation and how the team controlled for the sources of variation in the
conduct of the experiment.
Finally, before actually performing the experiment the group must
decide how the responses will be measure, what task each member of the
team will perform, and the exact technique to be used in dropping the
helicopters. Plus handling any other concerns related to the conduct of the
experiment that the team discussed during the planning of the experiment.
Subsampling vs. Replication
Recall, in the discussion of one factor experiments we introduced the
concept of replication vs. subsampling. Replication of an experimental trial
involves applying those experimental settings to a new experimental unit
(EU). In our case, the EU is an individual paper helicopter. So, replication
of a trial requires building a new paper helicopter with the same settings.
Subsampling occurs when we take multiple observations on the same EU, so
in our case the same helicopter would be dropped multiple times during a
single experimental trial and the response measured for each drop; the
multiple drops of the same helicopter do not constitute replication.
4
For this experiment, it is up to the teams to decide whether or not they
wish to employ subsampling. Individual helicopter drops can be quite
variable, so making multiple drops of each individual helicopter may be
advantageous to achieving better experimental results. If the team decides to
employ subsampling, simply add multiple columns to the data table for each
response and record the subsample observations in those columns for each
experimental trial. If you do incorporate subsampling in the experiment,
then I recommend that you use the average of the subsample drops for each
trial; this keeps the analysis straightforward. Remember, subsampling is not
replication, so you still need to build your replicate helicopters as directed in
the instructions. In your write-up please specify whether or not subsampling
was employed. It is not a requirement, but is a good experimental practice
when feasible.
In your write up specifically state whether or not subsampling
(essentially dropping the same helicopter multiple times in the same
run) was employed.
Analysis of the Experimental Results
The analysis of the experimental data and optimization are to be
performed using the Fit Model platform in JMP. The following is the
sequence of steps to performing the analysis. You will need to do this for
each of the two responses keeping in mind that Flight Time (or the
calculated rate) is the by far the primary response. Analyze Flight Time
and Accuracy separately; do not analyze them simultaneously even though
JMP allows one to do so.
1. In the Fit Model launch window specify a full factorial model –
include the 3-way interaction.
2. In the Fit Model report window, using the Actual by Predicted plot
and Lack of Fit report determine if any curvature exists in the
relationship of the response to the factors. Clearly report on your
findings. Even if significant Lack of Fit appears present continue on
with the analysis. Using the Parameter Estimates table determine
the best model to predict the response. One can do this by looking at
p-values for the hypothesis tests and the relative magnitudes of the
estimated effects. In making decisions with regard to retaining terms
in the model, at this point we are enforcing the heredity principle. If
an interaction is significant or active then retain any lower order terms
that are incorporated in to the interaction. Once you specified a final
5
model be certain to carefully document your analysis and result.
3. Once you have found a best model for each of the responses, then save
the JSL script for the analysis (open the main report menu at the top of
the report window and to the left of Response, then click on the Script
submenu and then click on Save Script to Data Table).
4. Finally, once you have determined the best model for each response,
save the prediction formula for that response to the data table. Click on
the main report menu, click on the Save Columns submenu, and then
click on Prediction Formula at the top of that drop down menu.You
will now have two prediction columns, one for each response, added to
the original data table.
Optimizing Helicopter Performance
Once you have determined the best model to predict each response, it is
time to determine the optimum helicopter design. For this phase of the
project we will use the Desirability Functions located in the Prediction
Profiler (Optimization and Desirability → Desirability Functions) menu.
The goal is to minimize the rate of descent (we want the helicopter to stay in
flight as long as possible). The goal for accuracy depends upon how you
measured it. If you measured accuracy in deviations
from target (say positive and negative deviations), then your response goal is
to match a target of 0; however if you measured it in absolute deviation, then
the goal is minimize the deviation from target.
Since you have previously saved your prediction formulas for each
response to the data table – if you have not done this, then revisit the
previous section and review the required analysis tasks – we do not need to
use Fit Model. We will access the Profiler directly through the Graph menu
(Graph → Profiler). Once in the Profiler launch window, place the two
prediction formula columns in the Y box (the box only accepts prediction
formula columns as inputs); the formula columns are how JMP determines
the models associated with each response.
6
Once in the Profiler report window, then complete the following tasks.
Refer to the Two Level Design Parts 1 and 2 notes for help with the Profiler
and Desirability.
1. Click on the Profiler Menu and select Desirability Functions (as
shown above). The Desirability Profiler Column and row should now
be added to the Prediction Profiler display.
2. Next, in the Desirability Function display (last column in the row for
the Rate response) double click, then in the Response Goal window
that appears, change the goal to Minimize for Rate (whatever name
you use) and set the Importance = 3. Do the same for the Accuracy,
but base the goal on how you measured accuracy (see comment above)
and leave the Importance = 1. You are now ready for optimization.
3. Once both response goals have been properly set, it is time to find the
helicopter configuration that will achieve the best results in Rate and
Accuracy. Click on the Prediction Profiler menu and select
Maximize and Remember from the drop down menu. JMP now
searches for helicopter design parameter settings that achieve the
response goals. Be certain to carefully document your Prediction
Profiler display and optimum settings in your write up.
Provide the Prediction Profiler Report, the results of the additional
helicopter drops, and a brief analysis of whether or not you were able to
improve the performance. You have now determined the settings of the
three factors that achieve the predicted optimum results for your two
responses. However these are predictions and the settings need to be
validated in practice.
Validating the Results
In the previous section we have determined the settings of our three
experimental factors that predict the optimum helicopter performance.
However, it is always a sound experimental practice to validate or confirm
that these results can be achieved in practice. In this section we now
validate our experimental results.
7
1. Build three helicopters to the optimum settings determined in the
previous section. It is important to build three distinct helicopters (EUs)
and not drop the same helicopter multiple times.
2. Using the same technique you used for the original experiment drop
each helicopter and measure the Rate and Accuracy. Be certain to
report on the results in your write –up.
3. Finally, discuss whether or not the results of the three drops confirm or
validate the results of your analysis in the previous section. In
otherwords did the observed Rate and Accuracy values come reasonably
close to the predicted target values? If the results do not seem to confirm
(this can happen), then discuss what might be the causes. There is no
absolutely right or wrong answer here.
Projected Potential Improvement
Often the analysis of the experimental results suggests that even better
performance can be achieved by extending the original ranges of the
experimental factors. Most likely the settings for the three experimental
factors for optimal performance (the most Desirable performance) occurred
on the boundaries of the original experimental region. The Prediction Profiler
is a very nice tool to visualize what direction one might move out of the
original experimental region in order to achieve even better performance.
As final stage of this project, use the Prediction Profiler to extend the
ranges of the three factors. You can even click on the axes at the bottom of
the Prediction Profiler in order to get the Profiler to predict performance
beyond the original experimental region. Try building a few helicopters
with new settings outside of the original experimental region and see if you
are able to achieve even better performance. In general, we do not
extrapolate our models out of the original experimental region, because we
do not know if our estimated models continue to work beyond that region.
However, if feasible it is a good practice to try and expand the ranges to
achieve better performance, then perform some additional experimental
trials and see if better performance is achieved. So in this section try some
extended ranges for the three factors and report on the observed results.
Were you able to improve performance?
8
Format of Final Report
The final report should be concise and to the point. Do not include
extraneous data tables and JMP output in the body of the report. The final
written report should be around 2 to 3 pages and may include a brief
appendix if you wish to include pictures, a JMP data table, etc. Any JMP
output used to support your findings should be copy and pasted into the
body of the report near the location where it is referenced. As an example, if
you are referring to a Prediction Profiler display, then copy and paste that
display (not the entire JMP report window) into the location in your report
where the reference is made. Please do not simply include JMP output at the
end of the report; this makes grading next to impossible. Remember the
Selection cursor tool in JMP can be used to copy and paste selected output
into other documents; it is also acceptable to use standard screen capture
software. I am not defining a specific line spacing, font size, margin, etc.;
with this exception, Wingdings are not acceptable as a font. Readably of the
report is the key.
III The Paper Helicopter Design
Figure 1 Picture of a Typical Paper Helicopter
9
Figure 2 Layout Diagram for a Paper Helicopter
Rotor
Width
Length Direction
Cut Along Center Line
Rotor
Bevel
Rotor
Length
Fold
Forward
Fold
Backward
Block
Bevel
Strut slits
Helicopter
Block
Paper Clip
Location
Helicopter
Nose
Width Direction
10