Comparison of two meteor trajectory solvers on the 2020 Perseid shower

By Eugene J. Mroz, Denis Vida and Paul Roggemans

Abstract: The Global Meteor Network trajectory solver obtains higher initial and geocentric velocities for identical Perseid meteors than the UFO solver of SonotaCo. Also, the average velocity for the GMN is slightly but statistically significant higher than for the UFO solver. The higher velocity explains a higher eccentricity for the orbits obtained by GMN. In spite of the lower velocity for the UFO solver, it has statistically significant higher beginning heights for these meteors while the ending heights are comparable. The length of the trajectories seems longer for the UFO solver than for the GMN while the durations are comparable. The differences between both solvers cannot be explained unless insight is provided in the computation method of the UFO solver.

 

1 Introduction

The Perseid meteor shower of August 2020 provided an opportunity to compare the results of two different meteor trajectory solvers and demonstrate anew that the frequency of meteor occurrence is an independent process that follows a Poisson distribution. The two trajectory solvers are UFO Orbit (v2.62) which is product of SonotaCo  and the Monte Carlo solver (Vida et al. 2020) used by the GMN solver. The trajectory variables selected for comparison are velocity, eccentricity, beginning and ending heights, trajectory length and duration.

 

2 Trajectory Solvers

Each of these trajectory solvers, hereafter referred to as UFO and GMN, uses different methods for computing meteor velocities.

The GMN solver is an open-source trajectory solver based on the lines of sight approach by Borovička (1990), but includes meteor dynamics as an additional constraint, as suggested by Gural (2012). In contrast to Gural (2012), the method does not impose an empirical velocity model to the trajectory.

The UFO solver is a copyrighted and proprietary product of SonotaCo¹.  The user manual (available on the website) states that the algorithms used in UFO Orbit are mostly based on the document by Hasegawa and Koseikaku (1983).   An internet search for this document was unsuccessful.

 

3 Data Source

The data for this study are taken from Perseid meteor observations made by the New Mexico Meteor Array during August 2020.  The New Mexico Meteor Array is part of the Global Meteor Network (GMN)².  It consists of 23 stations that record the night skies above an area of about 40000 sq. km centered on Albuquerque, NM.  A station consists of a Raspberry Pi single board computer, a low light level video camera and the RPi Meteor Station (RMS) software package for meteor detection.  This system records and analyzes the video and extracts video clips of detected meteors that are archived and uploaded to the GMN server.  The GMN server finds meteors which were observed by more than one station, computes the trajectories and calculates the orbits. In addition, the RMS software produces a data record of each meteor observation suitable for analysis by the freely available SonotaCo UFO Orbit (v2.62) software package.

From the August 2020 observations, 1031 Perseid meteor trajectories as computed by both GMN and UFO were deemed to be of the same event. This was done by selecting Perseid meteor trajectories from each methodology with trajectory start times within 1 second and a radiant separation angle of less than 5 degrees.  After removing 29 pairs as outliers (more than 3 standard deviations from the mean velocity, height, length and duration), 1002 trajectory pairs were retained for statistical analysis using SAS University Edition software package Version 3.8 (Basic Edition).

 

4 Approach

To compare the results of the GMN and UFO algorithms on the same set of 1002 meteor observations, we use a statistical procedure called a paired sample t-test.  In this test, each subject (in this case, each meteor observation) is measured twice (a trajectory is calculated by GMN and UFO), resulting in pairs of observations (pairs of trajectory solutions). The paired sample t-test determines whether the mean difference between the two sets of observations (two sets of trajectory solutions) is zero. The null hypothesis assumes that the true mean difference is zero and the t-test tests whether observed departures from zero are statistically significant.  A test statistic (t-statistic) is calculated from the mean and standard deviation of the data which is then compared to a theoretical distribution of the t-statistic derived from a normal distribution.  The probability of obtaining the t-statistic under the null hypothesis at a specified degree of confidence is then calculated which is used to judge whether the results provide sufficient evidence to reject the null hypothesis.  For this study we chose to use an alpha of 0.001 which corresponds to a 0.1% chance (or less) that the obtained result (that is, the mean difference of the two variables being compared) could happen by chance if the null hypothesis was true.  By choosing a low value of alpha, we are setting a high bar so that there is a low probability that the measured difference happened by chance.

 

5 Velocity

Geocentric velocity is the velocity a meteor would have in the absence of the Earth’s gravitational attraction.  It is a function of the Earth’s escape velocity and the initial velocity and altitude of the meteor when observed.

 

GMN found the mean geocentric velocity v_g to be 58.6 km/sec compared with 56.8 km/sec by UFO. The 99.9% confidence limits of both means are ±0.2 km/s.  These are both significantly lower than the accepted value of 59.1 km/sec (Jenniskens et al., 2016).

Figure 1 shows the distribution of the differences between the geocentric meteor velocities as computed by GMN and UFO on the same set of Perseid meteors. A paired t-test shows that GMN calculates the mean geocentric velocity to be 1.75 km/s faster than calculated by UFO (Table 1, Δ GMN_Vg – UFO_Vg).

 

 

As expected, this difference mirrors a difference of similar magnitude for the initial velocity used by each method. Figure 2 shows the distribution of the differences in initial meteor velocities as computed by GMN and UFO. A paired t-test shows that the initial velocity as calculated by GMN is 1.71 km/s faster than the initial velocity used by UFO (Table 1,  Δ GMN_Vinit – UFO_Vo).

We believe this difference is the result of a difference in how initial velocity is determined.  The GMN solver uses the first 40% of the data points along the trajectory to estimate the initial velocity and includes a deceleration term (Vida et al., 2020).   Whereas UFO uses the average velocity over the entire track of the meteor.  GMN also computes the average velocity so it is interesting to compare this to the average velocity computed by UFO.  Figure 3 shows the paired t-test for the average velocity as calculated by GMN is 0.43 km/s faster than that calculated by UFO (Table 1, Δ GMN_Vavg – UFO_Vo).  The reason for this difference is unclear.

 

Table 1 – Summary statistics from paired t-tests for 1001 meteor trajectories solved by GMN and UFO.  Mean, is the average of the variable. Std. Dev. is the standard deviation of the variable. Std. Err. is the estimate standard deviation of the sample mean. Min., is the minimum value. Max., is the maximum value. LCL/UCL Mean, are the Lower and Upper 99.9% confidence limits of the mean. LCL/UCL Std. Dev. are the Lower and Upper 99.9% confidence limits of the standard deviation. t-value, or the t-statistic, this is the ratio of the mean of the difference in means to the standard error of the difference. Pr > |t|, here the p-value is the two-tailed probability computed using the t-distribution.  If the p-value is less than the specified alpha level (0.001), then the difference is significantly different from zero.

Difference (Δ)	Mean	Std. Dev.	Std. Err.	Min.	Max.	LCL Mean	UCL Mean	LCL Std. Dev.	UCL Std. Dev.	t-Value	Pr > |t|
Δ GMNVinit – UFOVo (km/s)	1.71	1.78	0.06	–4.27	8.25	1.52	1.89	1.66	1.92	30.29	<0.0001
Δ GMNVg – UFOVg (km/s)	1.75	1.82	0.06	–4.33	8.47	1.56	1.94	1.69	1.96	30.55	<0.0001
Δ GMNVavg – UFOVo (km/s)	0.43	1.56	0.05	–5.79	7.22	0.26	0.59	1.46	1.69	8.63	<0.0001
Δ GMNecc – UFOecc	0.13	0.14	0.00	–0.35	0.64	0.11	0.14	0.13	0.15	29.61	<0.0001
Δ GMNHb – UFOHb (km)	–0.64	1.88	0.06	–9.88	13.14	–0.84	–0.45	1.75	2.03	–10.81	<0.0001
Δ GMNHe – UFOHe (km)	0.08	1.20	0.04	–7.42	9.60	–0.04	0.21	1.11	1.29	2.23	0.026
Δ GMNLen – UFOLen (km)	–1.18	3.07	0.10	–13.93	11.92	–1.50	–0.86	2.86	3.31	–12.18	<0.0001
Δ GMNdur – UFOdur (sec)	0.004	0.045	0.001	–0.189	0.203	–0.001	0.009	0.041	0.048	2.9	0.0038

 

6 Eccentricity

Eccentricity is a measure of how much an elliptical orbit deviates from a perfect circle and varies between 0 (circle) and 1 (parabola).  Eccentricity values greater than 1 are indicative of hyperbolic trajectories.  The Perseid meteors have a high eccentricity of 0.95 (Jenniskens et al., 2016) and it is of interest to compare the eccentricity as computed by both methods.

 

 

For the GMN solver, the average eccentricity was 0.92 and 19.7% of the 1001 jointly computed trajectories had computed eccentricities of greater than 1.  Whereas, the UFO solver had only 5.0% such trajectories but with a much lower average eccentricity of 0.79.  Figure 4 shows the distribution of the eccentricity Δs.  A paired t-test shows the eccentricity as calculated by GMN is 0.13 higher than that calculated by UFO (Table 1, Δ GMN_ecc – UFO_ecc).

Obtaining accurate velocity measurement is important for meteors like the Perseids that have highly eccentric orbits.  For the GMN solver, fast meteors such as the Perseids present a challenge because they result in fewer data points along the trajectory path from which to make an accurate velocity estimate.  So, whereas the GMN solver tends to generate more accurate and faster initial velocities, this can easily result in a higher percentage of computed trajectories with an apparent (but false) hyperbolic trajectory.  For the UFO solver, the lower average eccentricity and lower percentage of hyperbolic trajectory solutions are both probably a consequence of the lower velocity that results from using the average velocity over the entire meteor track.

 

7 Altitude

Both methods also solve for beginning and ending altitude of the observed meteor track. For the beginning altitude, the GMN solver is –0.64 km lower than the UFO solver
(Table 1, Δ GMN_Hb – UFO_Hb). Figure 5 shows the distribution of the Δs for beginning altitude.

For ending altitude, the GMN and UFO solvers give essentially the same result as can be seen Table 1 (Δ GMN_He – UFO_He) where the mean value is 0.08 km but the probability of it being zero is 0.026 which is higher than our specified alpha of 0.001. Thus, the lower and upper 99.9% confidence limits for the mean (–0.04 to 0.21 km) encompass zero.  Figure 6 shows the distribution of the Δs for ending altitude.

The altitudes of the highest and lowest stations in the network are 2247m and 1485m, respectively; a difference of 762m.  Perhaps the observed Δ reflects differences in how the two solvers handle the task of computing trajectories from sites with disparate altitudes.

It’s not clear why the two solvers should produce a larger Δ and larger spread of values for the beginning altitude than for the ending altitude.

 

 

 

8 Meteor track length

Comparing the solutions of the two solvers for meteor track length also reveals Δs.  Track length is calculated as part of the output from the UFO solver but is not directly available from the GMN solver output.  However, it can be estimated from the latitude, longitude and height of the track beginning and end points.  This calculation was performed using the SAS function GEODIST which calculates the geodetic distance between the two coordinates and accounts for the curvature of the Earth (Vincenty, 1975).  To account for vertical as well as horizontal distance traveled by the meteor, the Pythagorean theorem was employed together with the SAS GEODIST function to calculate the length of the meteor track between the beginning and ending latitude, longitude and altitude.

The results (Table 1, Δ GMN_Len – UFO_Len) show the GMN average track length to be about 1.18 km shorter than calculated by UFO.  There seems to be a considerable number of observations skewed to the negative side of the distribution of the Δs (Figure 7).  It is perhaps noteworthy that the camera system records 25 frames per second which is 0.04 seconds per frame.  At a nominal velocity of 59 km/sec, a Perseid meteor travels 2.4 km within a 0.04 second (single frame) interval.  So, the apparent Δ in trajectory length between the two methods is equivalent to about half of the exposure time of a single frame (0.02 seconds) which is below the temporal resolution capability of the meteor detection system.  This may be a situation where the statistically significant Δ does not necessarily accurately signify a significant difference in the performance of the two solvers.

 

 

 

9 Meteor track duration

The two solvers are in much better agreement when it comes to the duration of the meteor track.  The mean Δ is 0.004 seconds but the 99.9% confidence limits for the mean and the p-value shows that this is statistically indistinguishable from zero (Table 1, Δ GMN_dur – UFO_dur).  Figure 8 shows the distribution of the Δs.

 

10 Poisson distribution of Perseid meteors

Over suitably short time intervals, meteor events are independent and the arrival rate is constant.  Therefore, the distribution of meteor events within this interval is expected to follow a Poisson distribution.  We tested for this distribution using the observations of Perseid meteors from the entire GMN network.  We chose the two-hour interval of peak intensity (21^h00^m – 23^h00^m, 12 August 2020 UTC) during which 277 Perseid meteors were observed throughout the network.  These observations were then tabulated at 1-minute intervals.  Figure 9 shows the histogram of the number of Perseids observed per minute plotted as a fraction of the total (277) Perseids counted over two hours.  Overlaying the data is a plot of the Poisson probability mass function (PMF) fit to the data.  The goodness-of-fit can be judged by the chi-squared test which gives a result of p = 0.48 indicating that the data distribution is, as expected, statistically indistinguishable from a Poisson distribution.

 

 

11 Discussion

The most significant differences between the two solvers appears to be how they compute meteor velocity and eccentricity.  Unsurprisingly, the difference in how the initial velocity is estimated manifests itself as a difference in the apparent geocentric velocity and also affects the computed eccentricity of the orbit.   For the other trajectory characteristics, although the differences between the two approaches are generally small, they appear to be statistically significant. How these relate to differences in the two trajectory solving algorithms is not clear.   It would be immensely helpful to have better documentation of the algorithms used by UFO Orbit to better understand the similarities and differences in approach taken by these two methodologies.

 

Acknowledgments

This report would not have been possible without the generous commitment of personal time, effort and resources of the owner/operators of the stations that comprise the New Mexico Meteor Array: Pete Eschman (Coordinator), John Briggs, Ollie Eisman, Jim Fordice, Bob Greschke, Tim Havens, Bob Hufnagel, Ron James, Steve Kaufman, Bob Massey, Alex McConahay, Gene Mroz, Jim Seargeant, Eric Toops, Bill Wallace and Steve Welch.

We used the data of the Global Meteor Network which is released under the CC BY 4.0 license.

 

References

Borovička J. (1990). “The comparison of two methods of determining meteor trajectories from photographs”. Bulletin of the Astronomical Institutes of Czechoslovakia, 41, 391–396.

Gural P. S. (2012). “A new method of meteor trajectory determination applied to multiple unsynchronized video cameras”. Meteoritics & Planetary Science, 47, 1405–1418.

Hasegawa I., Koseikaku K. (1983). “Tentai Kidou Ron (Determination of Orbits)”. ISBN 4-7699-0572-6 C 3044.

Jenniskens P., Nénon Q., Albers J., Gural P. S., Haberman B., Holman D., Morales R., Grigsby B. J., Samuels D. and Johannink C. (2016). “The established meteor showers as observed by CAMS”. Icarus, 266, 331–354.

Vida D, Gural P., Brown P. G., Campbell-Brown M., Wiegert P. (2020). “Estimating trajectories of meteors: an observational Monte Carlo approach–I. Theory”. Monthly Notices of the Royal Astronomical Society, 491, 2688–2705.

Vincenty T. (1975). “Direct, and Inverse Solutions of Geodesics on the Ellipsoid with Application of Nested Equations”. Survey Review, 22, 88–93.