Performance Verification Statement for Systea WIZ Probe Phosphate Analyzer.

Abstract

The Alliance for Coastal Technology (ACT) conducted a sensor verification study of in situ nutrient analyzers during 2016 to characterize performance measures of accuracy, precision and reliability. The verification including a week of laboratory testing along with three moored field deployments in freshwater, estuarine, and oceanic coastal environments. Laboratory tests of accuracy, precision, and range were conducted at the University of Maryland’s Chesapeake Biological Laboratory (CBL) in Solomons, MD. A series of five tests were conducted to evaluate performance under controlled challenge conditions including: concentration range, temperature, salinity, turbidity, and dissolved organic carbon. All laboratory tests were conducted in 250 L polypropylene tanks using RO water as the initial matrix, within a temperature controlled room. Instruments sampled from a common, well-mixed, test tank maintained at a documented level of known challenge condition. Instruments were set-up by the manufacturer daily prior to the start of each individual laboratory tests, exposed to each test condition for a period of three hours, and programmed to sample at a minimum frequency of 30 minutes. Reference samples were collected every 30 minutes for five timepoints during corresponding instrument sampling times for each test. For the laboratory concentration range challenge the absolute difference between the Systea-PO4 and reference measurement across all timepoints for trials C0 – C5 ranged from 0.0067 to 0.0277 mgP/L, with a mean of 0.0041 ±0.0091 mgP/L. There was a small but significant increase in the measurement difference with increasing concentration as determined by linear regression (p=0.02; r2=0.20). However, the change in accuracy mostly occurred at the highest two test concentrations (0.406 and 1.87 mgP/L) with absolute measurement errors of 0.022 and 0.171 mgP/L, respectively. An assessment of precision was performed by computing the standard deviations and coefficients of variation of the five replicate measurements for C1 – C5 concentration trials. The standard deviation of the mean ranged from 0.0005 to 0.0040 mgP/L across the five trials, and the coefficient of variation ranged from 0.93 to 4.75 %. For the laboratory temperature challenge with testing at 5 oC, the absolute difference between instrument and reference measurement across all timepoints for trials C2 – C4 ranged from -0.0026 to 0.0170 mgP/L, with a mean of 0.010 ±0.005 mgP/L. Measurement differences were significantly higher at 5 oC versus 20 oC for each of three concentration test with the increased offset being 0.007, 0.005 and 0.017 mgP/L greater, respectively, for C2, C3, and C4. Similar to test results at 20 oC, there was a much larger offset at the C4 level compared to the lower two concentrations for the 5 oC test. For the laboratory salinity challenge performed at the C3 concentration level, the absolute difference between instrument and reference measurement across all timepoints for the three added salinity levels ranged from 0.0030 to 0.0333 mgP/L, with a mean of 0.0205 ±0.0078 mgP/L. There was a statistically significant response to increased salinity with measurement offsets increasing (over-predicted) as salinity increased. A linear regression of the measurement differences versus salinity was significant (p<0.001; r2=0.86) with a slope of 0.0009 and intercept of -0.0010. The average offset at salinity 30 was around 0.028 mgP/L higher than for the zero salinity trial which corresponded to a relative error of approximately 90%. For the laboratory turbidity challenge, performed at the C3 concentration level, the absolute difference between instrument and reference measurement across all timepoints for the two added turbidity levels ranged from -0.0208 to 0.0046 mgP/L, with a mean of -0.0079 ±0.0091 mgP/L. A linear regression of the measurement differences versus turbidity was significant (p<0.01; r2=0.59), with a slope of -0.0008 and intercept of 0.004, however the trend line was clearly forced by the large decrease at 100 NTU and test results should not be interpreted to suggest a strong predictable relationship. For the laboratory DOC challenge, pabsolute difference between instrument and reference measurement across all timepoints for the two added DOC levels ranged from -0.0018 to 0.0100 mgP/L, with a mean of 0.0018 ±0.0034 mgP/L. A linear regression of the measurement differences versus DOC concentration was not significant (p=0.71; r2=0.01). A 32 day deployment occurred from May 26 through June 27 in the Maumee River, at the facilities of the Bowling Green, Ohio Water Treatment Plant. The Systea-PO4 operated successfully during the entire 32 day deployment sampling at 15 minute intervals. The Systea-PO4 generated 3031 observations out of a possible 3052 for a data completion result of 99.3%. During the deployment 21 data points were not reported by the instrument. The average and standard deviation of the measurement difference between instrument and reference PO4 measurements for each matched pair (n=50 of a possible 51 observations) over the total deployment was 0.004 ± 0.024 mgP/L with a total range of -0.086 to 0.035 mgP/L. There was no significant trend in measurement difference over time as estimated by linear regression (p= 0.86; r2=0.001). A linear regression of instrument versus reference measurement was significant (p<0.01; r2 = 0.25) but with a slope of only 0.28 and intercept of 0.019. Measurement accuracy clearly declined when concentrations exceeded 0.05 mgP/L. An 84 day moored field test was conducted in Chesapeake Bay from July 18 to October 10, 2016. The Systea-PO4 operated continuously for the period of its deployment sampling at 30 minute intervals but was retrieved 12 days prior to the scheduled end date of the deployment to send to the next field test in HI. While the unit was deployed it reported 3086 of a possible 3402 accepted values for a data completion result of 90.7%. During its operation, 64 values were flagged by the instrument with no data, and 252 were omitted as outliers when reported values exceeded 10 times above observed levels or were less than -0.004. The average and standard deviation of the measurement difference between instrument and reference PO4 measurements for each matched pair (n=78 of a possible 103 observations) over the total deployment was 0.019 ±0.014 mgP/L, with the total range of differences between -0.019 to 0.049 mgP/L. There was no significant trend in measurement difference over time as estimated by linear regression (p=0.58; r2=0.004) over the deployment period. A linear regression of the instrument versus reference measurements was not significant (p=0.58; r2 = 0.004) and the instrument generally over-predicted concentrations. A one month long moored field test was conducted in Kaneohe Bay from October 3, 2016 to November 2, 2016. The Systea-PO4 was deployed on day four of the field test and operated successfully for the remaining 26 days of the deployment. During the deployment the Systea-PO4 returned 2477 acceptable instrument measurements of a possible 2496 measurements for a data completion result of 99% (13 data points were not reported and 6 observations were omitted as outliers having values >10x maximum reference). The average and standard deviation of the differences between instrument and reference readings over the entire deployment (n=63 out of a possible 63) were -0.0004 ± 0.002 mgP/L, with a total range of -0.0058 to 0.0044 mgP/L. There was a small but statistically significant trend in the measurement difference over time (p=0.0003; r2 = 0.247) during the deployment, with a slope of 0.0001 mgP/L/d. A linear regression of the instrument versus reference data was highly significant but with a low regression coefficient (p=0.001; r2 = 0.209). The regression had a slope of 0.999 and y intercept of -0.0004. Under and over predictions were fairly evenly distributed