Performance Verification Statement for NOC Nitrate Analyzer.

Abstract

The Alliance for Coastal Technology (ACT) conducted a sensor verification study of in situ nutrient analyzers during 2016 to characterize performance measures of accuracy, precision and reliability. The verification including a week of laboratory testing along with three moored field deployments in freshwater, estuarine, and oceanic coastal environments. Laboratory tests of accuracy, precision, and range were conducted at the University of Maryland’s Chesapeake Biological Laboratory (CBL) in Solomons, MD. A series of five tests were conducted to evaluate performance under controlled challenge conditions including: concentration range, temperature, salinity, turbidity, and dissolved organic carbon. All laboratory tests were conducted in 250 L polypropylene tanks using RO water as the initial matrix, within a temperature controlled room. Instruments sampled from a common, well-mixed, test tank maintained at a documented level of known challenge condition. Instruments were set-up by the manufacturer daily prior to the start of each individual laboratory test, exposed to each test condition for a period of three hours, and programmed to sample at a minimum frequency of 30 minutes. Reference samples were collected every 30 minutes for five timepoints during corresponding instrument sampling times for each test. For the laboratory concentration range challenge the absolute difference between the NOCNO23 and reference measurement across all timepoints for trials C0 – C5 ranged from -1.3061 to 0.0234 mgN/L, with an overall mean of -0.314 ±0.445 mgN/L. There was significant trend in instrument offset versus concentration as estimated by linear regression (p=0.0006; r2=0.39). The trend was driven by the substantially higher offsets at the C4 and C5 test concentrations (approximately 5 mgN/L) where the measurement error approached 20%. An assessment of precision was performed by computing the standard deviations and coefficients of variation of the five replicate measurements for C1 – C5 concentration trials. The standard deviation of the mean ranged from 0.002 to 0.040 mgN/L across the five trials, and the coefficient of variation ranged from 0.36 to 12.9 %. For the laboratory temperature challenge with testing at 5 oC, the absolute difference between instrument and reference measurement across all timepoints for trials C2 – C4 ranged from -0.629 to 0.056 mgN/L, with a mean of -0.048 ±0.194 mgN/L. The measurement difference at C2 was not significantly different between temperatures; however, the offset at C3 was significant greater at 5 oC then at 20 oC (0.032 vs. 0.003 mgN/L). Only one timepoint comparison was generated for the C4 trial so no statistical comparison was possible, however the greater negative offset was similar to test results at 20 oC. For the laboratory salinity challenge performed at the C3 concentration level, the absolute difference between instrument and reference measurement across all timepoints for the three added salinity levels ranged from -0.281 to 0.021 mgN/L, with a mean of-0.155 ±0.086 mgN/L. A linear regression between salinity and measurement error was not significant (p=0.17; r2=0.11), however, there was a noticeable increase in measurement variability and concentrations were consistently under-predicted at each added salinity level compared to zero. For the laboratory turbidity challenge, performed at the C3 concentration level, the absolute difference between instrument and reference measurement across all timepoints for the two added turbidity levels ranged from 0.010 to 0.050 mgN/L, with a mean of 0.030 ±0.016 mgN/L. A linear regression of the measurement differences versus turbidity was not significant (p=0.15; r2=0.15). For the laboratory DOC challenge, performed at the C3 concentration level, the absolute difference between instrument and reference measurement across all timepoints for the two added DOC levels ranged from -0.086 to 0.009 mgN/L, with a mean of 0.039 ±0.042 mgN/L. A linear regression of measurement differences versus DOC concentration was highly significant (p<0.0001; r2=0.79), with a slope of -0.004 and intercept of 0.029. The measurement offset was approximately 0.08 more negative at 10 mg/L DOC compared to lab RO which corresponded to a relative error of approximately 8%. A 32 day deployment occurred fromwater which corresponded to a relative error of approximately 8%. A 32 day deployment occurred from May 26 through June 27 in the Maumee River, at the facilities of the Bowling Green, Ohio Water Treatment Plant. The NOC-NO23 operated during the entire 32 day deployment sampling at hourly intervals, but due to a faulty SD memory card, the data from 5/27 to 6/7 were lost and during the last 6 days of the deployment 122 values were flagged by the instrument as “low precision”. Overall, the NOC-NO23 generated 375 accepted observations out of a possible 763 for a data completion result of 49.1%. The average and standard deviation of the measurement difference between instrument and reference NO3 measurements for each matched pair (n=21 of a possible 51 observations) over the total deployment was -1.38 ± 1.29 mgN/L with a total range of -6.12 to 2.16 mgN/L. There was no significant trend in measurement difference over time as estimated by linear regression (p= 0.48; r2=0.027). A linear regression of instrument versus reference measurement was highly significant (p<0.001; r2 = 0.77) but with a slope of only 0.546 and intercept of 0.81. An 84 day moored field test was conducted in Chesapeake Bay from July 18 to October 10, 2016. The NOC-NO23 malfunctioned during the first 3 days of the deployment, and the manufacturer was given permission to exchange the instrument with a new unit but keeping the same reagent and standards originally prepared. The replacement instrument operated from 7/21 to 8/21, measuring at hourly intervals, but then also failed. The instrument returned 603 data point out of a possible 2012 for the entire deployment period, with 1359 points missing and 50 flagged with no result calculated. While the unit was deployed it reported 603 of a possible 653 values for a data completion result of 92.3% (but only 33% of the scheduled total deployment was achieved). The average and standard deviation of the measurement difference between instrument and reference NO3 measurements for each matched pair (n=47 of a possible 103 observations) over the total deployment was -0.005 ±0.010 mgN/L, with the total range of differences between -0.027 to 0.031 mgN/L. There no significant trend in measurement difference over time (p=0.85; r2=0.001). A linear regression of the data was highly significant (p<0.0001; r2 = 0.53), but with a slope of only 0.54 and intercept of 0.0009. A one month long moored field test was conducted in Kaneohe Bay from October 3, 2016 to November 2, 2016. The NOC-NO23 operated successfully for the entire 30 day deployment, sampling at hourly intervals, returning 720 measurements for a data completion result of 100%. The average and standard deviation of the differences between instrument and reference readings over the entire deployment (n=73 out of a possible 73) were -0.013 ± 0.007 mgN/L, with a total range in the differences of -0.0394 to -0.0029 mgN/L. There was a small but statistically significant trend in the measurement difference over time (p=0.0009; r2 = 0.182) during the deployment, with a slope of 0.0003 mgN/L/d. The NOC-NO23 under-predicted all measurements and a linear regression of instrument versus reference concentrations was not significant (p=0.13; r2 = 0.04).