MTP/ER2 MP-File T(z) versus RAOB T(z) for SOLVE
Bruce L. Gary, 2001.04.24

This page describes an analysis comparing T(z) profiles from the MTP/ER2 SOLVE MP-file archive with RAOB T(z) profiles.  The goal is to produce a plot of RMS differences versus altitude, and to therefore "validate" pre-mission predicted performance.  If, perchance, there was an unforseen oversight in the analysis, it will reveal itself by producing a discrepancy between the RAOB and MTP profiles.

The flight data naturally segregate themselves into two types:  ferry flight and science flight data.  The ferry flights afford more RAOB comparison opportunities, since they tend to be over land areas.  The science flights tend to be over oceans, affording fewer RAOB comparisons.  The gain calibration analysis indicates that the two flight data types may also differ in ways that are not fully understood at this time.  Therefore, I will perform separate analyses for ferry flights and science flights, and afterwards I will combine them.  Since there are so few science flight RAOB site comparisons, and since it is performance during the science flights that matters most, I will present detailed analyses of those few occasions that do exist.

All MP-files have creation dates of 2001.02.03.  All MMS data were checked with the archive versions on 2001.04.08 and the latest archive MM-files were used.  For ER000109, ER000111 and ER00014 the creation date was 2000.07.14 ; for ER000316 the MM-file creation date was 2000.07.20; for ER000318 the MM-file creation date was 2000.07.18.

There are three section in this web page:

    1.  Ferry Flight comparisons of flight level temperature biases of NAV, MMS and MTP using RAOBs as a standard,
    2.  Ferry Flight comparisons of MTP T(z) profiles with RAOB T(z) profiles,
    3.  Ferry Flight RMS Performance of MTP T(z) Profiles (using RAOBs as the standard),
    4.  Science Flight comparisons of MTP T(z) profiles with RAOB T(z) profiles, and
    5.  Science Flight RMS Performance of MTP Profiles (using RAOBs as the standard).
    6.  Are Russian RAOBs "Warmer" than American RAOBs?


Section 1:  Ferry Flight Comparisons of Flight Level Temperatures

SOLVE had five ER-2 ferry flights, which are summarized in the following tables.  Each table has a column for the 3-letter RAOB site designation, a time corresponding to flight close to the RAOB site (given in kiloseconds), ER-2 pressure altitude (including a -3.51 mb correction to the MMS altitude), RAOB interpolated temperature (interpolated in time, and in distance if more than one RAOB site is used), navigation system static air temperature (Tnav), MMS air temperature (Tmms), MTP temperaure (Tmtp, from MP-file in archive), and four columns for the difference temperatures Tnav-Traob, Tmms-Traob, Tmtp-Traob, and Tmms-Tmtp.  One purpose served by creating tables like these is that they permit estimates to be made of biases, with respect to RAOBs, of the navigation system static air temperatures, MMS static air temperatures and MTP flight level air temperatures.  All flight level air temperatures have been subjected to a 500-second running average so as to remove short spatial frequency fluctuations that are unwanted for the task of comparing flight level temperature with RAOB ascents.  The 500 second length is meant to approximate a distance of 1 degree of arc, which is my maximum distance allowed for accepting a ROAB site for inclusion in the comparison table.

 ER000109, CA to MA (DFRC, California to Westover, Massachusetts):
 Seq#  RAOB Site  UT [ks]  Zp [km]  Traob [K]  Tnav [K]  Tmms [K]  Tmtp [K]  dTnav [K]  dTmms [K]  dTmtp [K] Tmms-Tmtp [K]
 1  GJT  59.6  20.39  215.2  216.5   n/a  214.5  +1.3    n/a  -0.7  n/a
 2  DEN  60.9  20.24  213.0  217.0    214.9  +4.0    +1.9  
 3  LBF  62.8  20.55  212.0  216.3    214.4  +4.3    +2.4  
 4  ILX  67.1  20.86  211.1  214.3    212.3  +3.2    +1.2  
 5  ILN  69.2  20.63  210.0  212.1    210.3  +2.1    +0.3  
 6  PIT  70.4  20.70  210.0  213.0    211.0  +3.0    +1.0  
 Note:  MMS data stopped before the first RAOB comparison time.

 ER000111, MA to MA (aborted before reaching Atlantic Ocean):
 Seq#  RAOB Site  UT [ks]  Zp [km]  Traob [K]  Tnav [K]  Tmms [K] Tmtp [K]  dTnav [K] dTmms [K]  dTmtp [K] Tmms-Tmtp [K]
 1  GYX  49.2  16.64  218.3   pq  217.7      -0.6    
 2  CAR  50.8  18.78  216.3  218#1  216.6  216.8  +1.7  +0.3  +0.5  -0.2
 3  YZV  52.4  19.10  217.1  218.3#0.3  216.1  217.0  +1.2  +0.2   -0.1  +0.3
 4  YZV  56.3  19.51  216.3  216.7#1  217.3  215.5  +0.4  -0.2   -0.8  +0.6
 5  CAR  58.3  19.42  214.9  218.6#1  219.0  216.5  +3.7  +2.4   +1.6  +0.8
 6  GYX  60.0  18.54  216.2  pq  pq  217.9  pq  +2.8   +1.7  +1.1

 ER000114, MA to KIR (Kiruna, Sweden):
 Seq#  RAOB Site  UT [ks]  Zp [km]  Traob [K]  Tnav [K]  Tmms [K] Tmtp [K]  dTnav [K]  dTmms [K]  dTmtp [K] Tmms - Tmtp [K]
 1  GYX  46.1  15.39  223.3  222.0  225.0    -1.3  +1.7    
 2  CAR  48.0  18.74  221.3  220.8  220.9  220.0  -0.5  -0.4  -1.3  +0.9
 3  YZV  49.8  18.80  219.7  221.0  221.0  219.9  +1.3  +1.3  +0.2  +1.1
 4  YYR  51.8  18.93  218.4  219.4  219.3  218.8  +1.0  +0.9  +0.4  +0.5
 5  BGB/BGE  57.9  19.79  205.3  204.5  204.2  204.1  -0.8  -1.1  -1.2  +0.1

 ER000316, KIR to MA:
 Seq#  RAOB Site  UT [ks]  Zp [km]  Traob [K]  Tnav [K]  Tmms [K]  Tmtp [K]  dTnav [K]  dTmms [K] dTmtp [K] Tmms-Tmtp [K]
 1   YZV/YJT  52.3  19.87  218.6  216.8  216.6  216.4   -1.8  -2.0  -2.2  +0.2
 2  CAR  54.8  20.18  217.2  217.8  217.0  216.7  +0.6  -0.2  -0.5  +0.3

 ER000318, MA to CA:
 Seq#  RAOB Site  UT [ks]  Zp [km]  Traob [K]  Tnav [K]  Tmms [K]  Tmtp [K]  dTnav [K]  dTmms [K]  dTmtp [K]  Tmms-Tmtp [K]
 1  PIT  54.2  19.49  210.0  210.9  211.0  210.3  +0.9  +1.0  +0.3  +0.7
 2  DTX/ILN  55.8  19.87  210.1  211.0  211.0  210.5  +0.9  +0.9  +0.4  +0.5
 3  DVN/ILX  58.4  19.97  210.3  209.8  209.7  209.5  -0.5  -0.6  -0.8  +0.2
 4  OAX/TOP  61.1  20.22  208.2  210.7  210.5  210.5  +2.5  +2.3  +2.3  +0.0
 5  LBF/DDC  63.0  20.37  209.6  208.1  208.2  207.9  -1.5  -1.4  -1.7  +0.3
 6  DEN  65.0  20.47  207.3  209.0  209.1  208.8  +1.7  +1.8  +1.5  +0.3
 7  GJT/ABQ  66.5  20.69  208.7  207.5  207.3  207.3  -1.2  -1.4  -1.4  +0.0
 8  FSX  68.7  20.84  209.0  209.4  209.3  208.7  +0.4  +0.3  -0.3  +0.6

The following average dT-values were determined for these 5 flights:

    Tnav - Traob = +1.06 +/- 0.35 K (N=25)
    Tmms - Traob = +0.38 +/- 0.30 K (N=21)
    Tmtp - Traob = -0.07 +/- 0.26 K (N=25)
    Tmms - Tmtp  = +0.44 +/- 0.08 K (N=19)

This last item, Tmms - Tmtp, is compatible with the combination of "Tmms - Traob" and "Tmtp - Traob" but since "Tmms - Tmtp" does not have Traob as an intermediary the accuracy of the MMS/MTP comparison is better. Neither dTnav nor dTmms correlate with Zp, or with Traob.  Tnav differs from Traob by a statistically significant 3 sigma, whereas Tmms and Tmtp do not differ significantly from Traob.  The goal of making Tmtp consistent with ROABs was apparently successful.  Although the direct comparison of Tmms with Traob shows only a 1-sigma difference, the fact that Tmms - Tmtp differ by a statistically significant amount suggests that one of these two must differ from Traob in a statistically significant amount, and Tmms is the most likely candidate considering its 1.3-sigma difference (compared with Tmtp's 0.3-sigma difference).

Section 2:  Ferry Flight Comparisons of MTP T(z) Profiles with RAOB T(z) Profiles

The next figure shows difference profiles, dT(z) = MTP's T(z) - RAOB's T(z), for all five ferry flights:

Figure 1.  Profiles of inidividual comparisons of MTP T(z) versus RAOB T(z) for all 5 ferry flights.

Three aspects of this figure deserve comment:

    1) the average profile does NOT have structure.  This means the "window correction table" used in reducing the data was good quality.
    2) MTP is warmer than RAOBs (this is true throughout the relative altitude region)
    3) there seems to be a slight trend, with MTP going from warmer-than to about-the-same-as RAOBs.

The sequence of altitude-averaged MTP-RAOB is tabulated here, and plotted below:

    ER000109    +0.66 +/- 0.21 K
    ER000111    +0.95 +/- 0.23 K
    ER000144    -0.08 +/- 0.34 K
    ER000316    -0.88 +/- 0.29 K
    ER000318    +0.36 +/- 0.16 K

Figure 2.  Trend of the profile average difference between MTP's T(z) and the RAOB T(z) for the 5 SOLVE ferry flights.

This trend is statistically NOT significant since the ratio of the slope to its SE uncertainty is only 1.1.  Therefore, the weighted average value is to be adopted, which is +0.35 +/- 0.10 K.  In other words, MTP profiles (averaged throughout the relative region -2.5 to +3.5 km) are warmer, on average, than RAOBs by 0.35 K, and this warmness is statistically significant, with a signal-to-noise ratio of 3.5.

Section 3:  Ferry Flight RMS Performace of MTP T(z) Accuracy and Comparisons With Prediction

The following figure shows the RMS difference between MTP/ER2 and RAOB.

Figure 3.  RMS performance of MTP/ER2 versus "relative altitude" (thick red trace), with average difference (dotted blue trace) removed.

According to this graph the MTP/ER2 differred from RAOBs near flight level by 1.5 K (after "correcting" for an average offset of 0.5 K).  This seems a little higher than expected.  Also unexpected is the lack of best performance near flight level.  An example of the shape of "predicted performance versus altitude" is shown in the next figure.

Figure 4.  RMS performance of MTP/ER2 versus "relative altitude" for SOLVE (red symbol and line trace) in relation to a predicted performance trace calculated for an earlier mission (POLARIS#3).  The meaning of the two predicted traces in explained in the text.

In predicting performance allowance must be made for a component of uncertainty associated with the temporal and spatial interpolations that are made when comparing MTP profiles with RAOB profiles.  If all comparisons could be made while flying directly over a RAOB site at the time the balloon was passing through the ER-2's altitude, then the dotted trace in the above figure would apply.  This dotted trace was computed during the process of deriving retrieval coefficients, which adopts a table for observable uncertainties.  The observable uncertainty table used in calculating retrieval coefficients and an associated predicted performance (shown in the figure) is given below:

EL# EL   Ch#1 S.E. Ch#2 S.E.

  1 +60.0  1.5 K  1.0 K
  2 +44.4  1.1 K  0.9 K
  3 +30.0  0.8 K  0.8 K
  4 +17.5  0.7 K  0.7 K
  5 + 8.6  0.7 K  0.7 K
  6   0.0 (see next paragraph)
  7 - 8.6  0.7 K  0.7 K
  8 -20.5  0.7 K  0.7 K
  9 -36.9  0.8 K  0.8 K
 10 -58.2  0.9 K  0.9 K

Since there are 10 elevation angles, and 2 observing frequencies, there are a total of 20 brightness temperatures.  However, the two horizon TBs will be averaged, producing OAT, so they will not show up as TB observables, explicitly.  We therefore will use 18 TB observables plus OAT, yielding a 19-observable input data vector.  For OAT adopt an a priori S.E. of 0.7 K.

I believe the component of uncertainty associated with comparing a set of MTP/ER2 profiles with RAOBs is approximately 0.75 K.  This component of uncertainty was derived for the DC-8 (described on another web page), and I shall assume it is the same for both the ER-2 and DC-8.  Adopting 0.75 K for this component of uncertainty allows for the derivation of a predicted performance when actual RAOB comparisosn are used, and this is what's shown by the thick solid trace labelled "INCL RAOB SE" in the legend in the above figure.

The MTP/ER2 SOLVE observed performance trace does not have the expected minimum near flight level.  I am puzzled by this!  The DC-8 data that was subjected to this kind of analysis had the expected minimum near the average flight level (an absolute altitude scale was used), and the RMS performance within a 4 km region near this level was beter than 1.1 K).  This issue should probably be studied further, to at least understand "what's going on."

The apparent RMS performance for SOLVE away from flight level is similar to the POLARIS predicted performance, which implies that one might expect that if the present analysis had extended beyond the relative altitude range of -2.5 to +3.5 km the observed RMS performance would be approximated by the POLARIS trace.  Thhis could not be done, since the MTP data in the MP-files were restricted this relative altitude range.  In light of the present analysis I would recommend that any future MP-file creations, should they occur, be modified to include a wider relative altitude region, such as from -6 to +6 km (where RMS performance is predicted to be better than 2.0 K).

Section 4:  Science Flight Comparisons of MTP's T(z) With RAOB T(z)

Since the science flights provide only a few opportunities for comparison with RAOBs I shall describe each of them individually.

ER000120

This flight is from Kiruna to essentially the North Pole, by way of Spitzbergen. Only one RAOB site was useable (the one over Spitzbergen had balloon bursts at low altitudes).

Figure 5First science flight MTP/RAOB comparison.  Thick red traces are MTP northbound (solid) and southbound (dotted). Green and blue traces are RAOB (at the time of northbound passage over the RAOB site) and after both MTP data times.  The horizontal black line corresponds to the ER-2 altitude during the northbound passage (thick red trace).  For the southbound MTP data (red dotted trace) the ER-2 was at 20.2 km (not shown).  Symbols "M" and "N" are used to represent MMS and NAV temperatures at flight level.

This pair of events are difficult to interpret.  First, the balloons burst at low altitudes (20 and 19 km), and there's a cooling trend (above 17 km) that grows with altitude (to at least 19 km).  The in situ air temperatures confirm the cooling trend (above 18.5 km) during the shorter 4.6 hour interval of MTP data.  Nevertheless, since the first RAOB (green trace) occurs at essentially the same time as the first MTP profile (thick red trace), there appears to be a temperature offset of about 1 K (MTP colder than RAOB).  The difference between the two MTP profiles (separated by only 4.6 hours) probably is real.  Profile difference data can only be obtained for altitudes below flight level (they will be included in a later part of this web page's analysis).  Because of the ambiguities of this event I will only include the northbound comparisons below flight level in the MTP RMS performance analysis (next section).

ER000127

This flight is over Russia, and only 1 RAOB site has data for the "before" and "after" times reaching altitudes of 21 km or higher.

Figure 6.  MTP (thick red) and RAOBs before (dotted green) and after (thin blue line) for 37.4 ks.

There's a large offset between the MTP T(z) profile and the two RAOBs.  MMS data is clearly discrepant with the RAOBs, since Tmms = 194.8 K at this time.  Considering that MMS may need an additional 0.38 K downard correction, its discrepancy is actually larger, being 4.6 K (consistent with Tmtp).  Either the air temperature became colder by 4.6 K after the first RAOB and quickly warmed the same amount by the time of the second RAOB, or the RAOBs used at this site on this date are poorly calibrated.  If their calibration is poor, then maybe the offset can be ignored, and the shape can be taken seriously.  If so, then MTP did OK.  I will decide later whether to include this comparison in a later MTP RMS performance analysis.

ER000131

RAOBs are missing or don't go high enough at the two sites that are close to the flight track (ENBJ and ENAS).  Too bad.

ER000202

RAOBs are available at only one site, EFSO (Lat 67.37, Lon 26.65), which is in Finland.

Figure 7.  Altitude profiles T(z) for MTP (thick red) and RAOBs before (dashed green) and after (thin blue line) the ER-2 passed RAOB site EFSO.

The shape of this MTP profile agrees with RAOBs, but, as with the previous comparison, the RAOBs are warmer than MTP.  In this case, the ROABs are only 2.0 K warmer than MTP at flight level.  At this time MMS was 1.0 K warmer than MTP, so the RAOBs are warmer than MMS by 1.0 K.  It appears that the MTP calibration procedure is inadequate at the beginning of a flight (which this is), because in many cases it produces temperatures that disagree with MMS by about 1 K for the first half hour.  In this case MTP is colder than MMS by 1.0 K until a half hour later.  I am reluctant to include the MTP versus RAOB difference profile in a statistical analysis, but since it meets my criteria of being "in the archive" as an MP-file, and the event is sufficiently close to the RAOB site, I will include it in the statistical analysis tabulation (in Section 5).

ER000203

Flight ER000203 has two times when a RAOB site was within 2 arc degrees of the flight track, and these two events are displaye4d in the following figures.

Figure 8.  Altitude profiles T(z) for MTP (thick red) and RAOBs before (dashed green) and after (thin blue line) the ER-2 passed RAOB site EFSO.  The thick greent race is a temporal interpolation of the two ROAB r\traces, and can be directly compared with the MTP trace.
 

Figure 9.  Altitude profiles T(z) for MTP (thick red) and RAOBs before (dashed green) and after (thin blue line) the ER-2 passed RAOB site EFSO.  The thick greent race is a temporal interpolation of the two ROAB r\traces, and can be directly compared with the MTP trace.

For this flight both MTP/RAOB comparisons show agreement, without a "MTP cold offset."

ER000226

This flight track went by several RAOB sites, but only two sites had RAOB coverage, permitting three MTP/RAOB comparisons.

Figure 10.  Altitude profiles T(z) for MTP (thick red) and RAOBs before (dashed green) and after (thin blue line) the ER-2 passed RAOB site ENBO southbound.  The thick green trace is a temporal interpolation of the two ROAB traces, and can be directly compared with the MTP trace.

Figure 11.  Altitude profiles T(z) for MTP (thick red) and RAOBs before (dashed blue) and after (thin green line) the ER-2 passed RAOB site ENBO northbound.  The thick green trace is a temporal interpolation of the two RAOB traces, and can be directly compared with the MTP trace.  Since the "before" RAOB did not go above 20.8 km, the comparison analysis will terminate at this altitude.

There's a whopping 2.8 K "MTP colder than RAOB" effect in this comparison event.

Figure 12.  Altitude profiles T(z) for MTP (thick red) and a "before" RAOB (dashed green).  Thre was no "after" RAOB, but since the "before" RAOB sampled the air only 1.5 hours before the ER-2 flew by this single RAOB will provide for an adequate comparison.

For all three MTP/RAOB T(z) comparisons the MTP is colder than the RAOB.  The next figure shows Tmms, Tmtp and Traob.

Figure 13.  Tmms (red), Tmtp (green) and Traob for the three comparison events treated in the previous figures.

The RAOBs in this figure are warmer than both MTP and MMS, but the first comparison event suffers from what appear to be MTP warm-up problems that are not adequately modeled by the gain equations.  For the middle comparison event the RAOB is very discrepant with both MMS and MTP, and for the third event the RAOB is somewhat discrepant with both.  I cannot account for these discrepancies.

ER000305

Only one RAOB site had adequate data, Finland's EFSO.

Figure 14.  T(z) for MTP (thick red) and RAOBs before (green dashed) and after (blue thin), and temporal-interpolated RAOB (thick green).

MTP gain is colder than the RAOBs.  This is the beginning of the flight, when MTP gains are not well established, accounting for a discrepancy of 1.0 K between MTP and MMS temperatures.  At 34.1 ks Tmms = 196.0 K, which is ~0.7 K colder than the RAOBs.  The shape of MTP's T((z) is in approximate agreement with the interpolated RAOB T(z).

ER000307

Flight ER000307 provides two RAOB comparison events, both near the beginning of the flight and both involving RAOBs from Finland.

Figure 15.  T(z) profiles for MTP (thick red), RAOBs before and after, and RAOB interpolated (thick green).

Figure 16.   T(z) profiles for MTP (thick red), RAOBs before and after, and RAOB interpolated (thick green).

Figure 17.  Tmms (red), Tmtp (green) and Traob (red ovals) for the early part of ER000307.

A pattern is preserved:  at the beginning of flights the MTP temperatures are colder than MMS, and MTP is colder than RAOBs.  At 26.3 ks Tmms = 197.4 K, which agrees with MMS.  By 27.5 ks MTP's gain equation has become appropriate and Tmtp agrees with both MMS and RAOB.

ER000311

There are four times when the ER-2 was close to RAOBs which got data close enough in time to the ER-2's passage to be useful.

Figure 18.  T(z) profiles for MTP (thick red), RAOB after (thin blue trace).  There as no "before" RAOB data but the "after" RAOB was close enough in time to the ER-2 passage to be useful.

Figure 19.  T(z) profiles for MTP (thick red), RAOB after (thin blue trace).  There as no "before" RAOB data but the "after" RAOB was close enough in time to the ER-2 passage to be useful.

Figure 20.  T(z) profiles for MTP (thick red), RAOB after (thin blue trace).  There as no "before" RAOB data but the "after" RAOB was close enough in time to the ER-2 passage to be useful.

Figure 21.  T(z) profiles for MTP (thick red), RAOB "before" (thin green) and 'after" (dotted green).  The "after" RAOB data extends to only 16.1 km; however, the ER-2 passage was close in time to the "before" RAOB which therefore provides an adequate RAOB comparison.

Figure 21.  Tmms (red trace), Tmtp (green trace) and Traob (red ovals) versus time.

Referring to Fig. 21 the RAOBs are all warmer than Tmms and Tmtp, which is the pattern for most of the science flights.

The previous 4 figures show that MTP's T(z) shapes are in agreement with the RAOB T(z).

ER000312

This is the last science flight, and three RAOB comparisons are possible.

Figure 22.  T(z) profiles for MTP (thick red), RAOBs before and after, and RAOB interpolated (thick green), for this flight's first comparison .

Figure 23.  T(z) profiles for MTP (thick red), RAOBs before and after, and RAOB interpolated (thick green), for this flight's second comparison .

Figure 24.  T(z) profiles for MTP (thick red), RAOBs before and after, and RAOB interpolated (thick green), for this flight's third comparison .

In all three cases the RAOBs are warmer than MTP (or MMS), which is shown more clearly in the next figure.

Figure 25.  Tmms, Tmtp and Traob (red ovals) during ER000312.

This concludes the detailed display of T(z) comparisons between MTP and RAOBs during the science flight portion of SOLVE.  An analysis of two aspects of these comparisons is taken up in the next two sections.


Section 5:  Science Flight RMS Performance of MTP T(z) Profiles

A procedure was followed similar to the one used to produce Fig. 1.   In this case, we wish to see the difference profiles, dT(z) = MTP's T(z) - RAOB's T(z), for the science flights:

Figure 26.  Difference plots, dT(z) = MTP's T(z) - RAOB's T(z), for the 18 cases in which RAOBs could be compared to MTP data during the science flights.

The following table summarizes the population average and SE for the above figure.

        dZp                Avg +/- SE                    Pop'n SE
   +3.5 km  -1.67 # 0.37 K    1.48 K
   +2.5     -1.75 # 0.31      1.27
   +1.5     -1.51 # 0.32      1.31
   +0.5     -1.41 # 0.26      1.11
   -0.5     -1.34 # 0.24      1.02
   -1.5     -1.19 # 0.22      0.92
   -2.5     -1.12 # 0.23      0.98

The entire average profile is colder than the RAOBs, averaging about -1.4 K.  Indeed, very few data in this plot are positive (e.g., 12 out of 122).  This is in stark contrast with the ferry flight counterpart, Fig. 1.  This difference is studied in a more straightforward way in the next section.

It is interesting to compare the population SE profile in the above figure to the population SE profile from the ferry flight RAOB comparisons (shown in Fig. 3).

Figure 27.  Comparison of two determinations for MTP's RMS performance in relation to a predicted RMS performance profile (for the POLARIS Mission).  For both the Ferry Flight case and the Science Flight case the average difference profile between MTP and RAOB has been ignored, meaning that the profiles shown correspond to the RMS performance if all the MP-file profiles were corrected by the average difference profile.

In this figure only the population SE is plotted, which corresponds to the hypothetical situation of correcting all MP-file data by the "average" profile.  This is a legitimate procedure, since in theory the removal of an average bias profile can be justified by claiming that a multitude of small calibration inaccuracies that were shared by a chunk of MTP data can only be detected by performing an analysis like the one in this web page, and users of the MP-file data deserve to have these small corrections applied to the exchange data.  For example, I performed this small empirical correction for the MTP/DC8 TOTE/VOTE data.  This final empirical correction procedure is meant to deal with such things as the use of retrieval coefficients from a RAOB data base that did not exactly represent conditions during the actual data flights, and errors in the derivation of window corrections.

I am unable to explain the odd shape of the observed profiles of MTP RMS performance in the above figure.  I have never seen this "misbehavior."  The best performance is about 2 km below flight level, not in the vicinity of flight level!  This is true for both the ferry flight data set and the science flight data set. Very odd!

What most bothers me, though, is the change in MTP offset (with respect to RAOB data) between the ferry flight data set and the science flight data set.  At the flight level altitude the two offsets are -0.07 +/- 0.26 K (ferry flights) and -1.37 +/- 0.25 K.  This is taken up in the next section.

Section 6:  Are Russian RAOBs "Warmer" than American RAOBs?

The science flights afforded 18 opportunities for comparing Tmms with Traob.  The average difference is:  Traob - Tmms = +0.81 +/- 0.24 K (population SE = 1.00 K).  Comparing this with the 21 comparisons of Tmms with Traob during the ferry flights yielded Traob - Tmms = -0.38 +/- 0.30 K (given above).  The difference between Traob/science and Traob/ferry is 1.19 +/- 0.38 K, with the science flight RAOBs being warmer.  This RAOB difference is statistically significant, since the difference dividied by the SE uncertainty is 3.1-sigma!

The next figure plots "Traob - Tmms" for the entire SOLVE Mission.

Figure 28.  "Traob - Tmms" for the entire SOLVE Mission.  The solid red squares are Science Flight comparisons, and the green stars are Ferry Flight comparisons.

Visually, the difference between the ferry flight and science flight comparisons is not very apparent.  Indeed, this figure leaves me unconvinced that there is any difference.  The one "outlier" (at location 13 - of course!) should be removed using Pierce's criterion (Pierce, 1879).  Doing this, and recalculating, yields:

    Traob - Tmms (Science Flights) = +0.62 +/- 0.16 K

Compared with Traob - Tmms (Ferry Flights) we obtain:

    Traob/science - Traob/ferry = +1.00 +/- 0.34 K.
 

This difference still has statistical significance, since the ratio of the difference to its uncertainty is 3.0-sigma.

So, are we to believe that the RAOBs in Russia, Finland and Norway are 1.0 K warmer than those over North America?

I don't know!

Before I'd believe that such a difference exists I'd want to see a more compelling case.   If MMS is to be used for determining such a difference there should be more comparisons than the 39 of this analysis (21 over North America and 18 over Russia, Finland and Norway).

If we are prepared to assume that there are no differences between the RAOBs of North America and those of Russia, Finland and Norway, then we are free to average the entire set of 38 MMS/RAOB comparisons to arrive at a new estimate of the MMS calibration.

    Tmms - Traob (all 39 RAOBs) = -0.31 +/- 0.19 K  (excluding the location 13 "outlier")

In other words, if all RAOBs were on the same scale, we would conclude that MMS is "cold" by about 0.3 K.  However, the uncertainty is almost as large as the difference, and this result is statistically not different from a zero difference.

Given the uncertainties of the RAOB comparisons of this web page's analysis, I recommend ACCEPTING the MMS temperatures as they exist in the archive!

Conclusion

It can be said that this web page analysis is "much ado about nothing" since one of its conclusions is a recommendation to accept the MMS temperatures as they exist in the archive.

A second result is that the MTP T(z) profiles are accurate to about 1.2 K RMS from -2.5 km to +2.5 km (shown in Fig. 27).  The shape of the RMS accuracy plot is not quite what was expected, and this is unusual.  I have no explanation for the slightly greater RMS uncertainties near flight level, or for the fact that accuracy appears to be better below flight level than at, and above, flight level.  In retrospect, it would have been better to include MTP T(z) data for an altitude range that extends farther below flight level than the range that was used, which extends farther above flight level.  It might be worth considering re-submitting MTP MP-files to the archive which cover the altitude region -4.0 to +3.5 km, for example.

____________________________________________________________________

This site opened:  March 22, 2001.  Last Update:  May 10, 2001