All programs › BMPFast › Validation and Verification
Testing Protocol for BMPFast
The guidelines, or set of rules, we follow when testing the BMPFast software.
The basis for testing
Testing is done using the information in the FDEP Environmental Resource Permit Applicant’s Handbook, Volume I, June 28, 2024. The BMPFast software must be consistent with the information in the Handbook.
For permitting agencies: the resolution of every item raised in testing and review, and the details behind it, are available to permitting agencies upon request. Email info@bmpfast.com.
Testing with past projects
The question is: does BMPFast replicate what was accepted in the past, as calculated with BMPTrains, as long as updated information from the Handbook is used? We use some of the example projects in the BMPTrains Manual, and then expand to other projects.
- The information and parameters for annual removal are set to be consistent with the required information in the Handbook. For example, the data must be consistent with the EMCs (Table 9.2, page 9.4), the annual rain volume (page 9.6, Appendix M) and the performance standards (pages 8.2–8.4). The mean annual runoff coefficient (ROC) must be calculated with the tables in Appendix N, and the annual mass loading with Equation 9.2 (page 9.5).
- The catchment areas must be divided into directly connected impervious areas (DCIA) and non-directly connected impervious areas (pervious areas), Table 9.1, page 9.2. As standard practice, a composite CN cannot be used for an area that includes DCIA. EMC values must be the same in both software packages when the comparisons are made.
- The effectiveness equations and the retention tables (Appendix O) are compared.
Unit, integration and hand-calculation testing
A unit test example uses Equation 9.2 of the Handbook, the average annual mass loading. The result is printed in BMPFast and can be compared with a hand calculation. Integration testing means checking the carry-over of data from one unit of code to another, such as the carry-over of the average annual runoff coefficient (ROC) into Equation 9.2. ROC is calculated from the tables of Appendix N and is treated as a “lookup” table.
Hand calculations that justify the choice of ROC must interpolate between values for combinations of non-DCIA CN and DCIA that are not always in units of 5. This makes most hand calculations challenging, so computer software — sometimes spreadsheets — is used for the “hand” calculations. Those “hand” calculations must frequently be validated as well. The Appendix O lookup tables for retention removal are another example: values fall between tables as well as within them.
To support this testing, BMPFast provides tools for ROC and retention removal, so results can be compared with other calculators or computer-assisted hand calculations (the ROC and Retention Efficiency Lookup Table tools in BMPFast). For complex calculations, tools are added to BMPFast for testing and discovery.
Final report testing ensures that the correct variables and populated data reach the printed and viewed final results.
Testing the improvements noted in the Handbook
BMPTrains was developed with the information accepted at the time (2020). Some information is now (2026) better defined, or new, and is listed in the Handbook. In addition to past projects, the following (not a complete list) are tested.
- Wet ponds without a littoral zone. BMPTrains assumed a wet pond has a littoral zone; corrections must be made for a pond without one. BMPFast provides the option of a pond with or without a littoral zone. In testing, the equations, their results and the input data are viewed to confirm the calculations are correct. Our action: we continue to show equation results and the data used for calculations. More are being added to BMPFast as further suggestions for testing and review are defined, and more can be provided for specific needs.
- BMPs in series (Equation 9.5, pages 9.6–9.7). The equation has limits on its use. It is not used for a series of retention BMPs in a catchment: the overall effectiveness does not use the stand-alone effectiveness of each BMP, and must be based on the cumulative retention. For wet detention ponds in series, the removal of all the ponds must be based on the cumulative average annual residence time. With multiple catchments, BMPTrains does not follow this cumulative retention or residence time; BMPFast does. The Handbook states: “This equation assumes each BMP acts independently of upstream BMPs, and that upstream BMPs do not impact performance of downstream BMPs.” Our action: promote education on the use of Equation 9.5. In a number of projects reviewed, the cumulative values were not used even within one catchment.
- Flow from one catchment’s BMP to the next. Retention and wet detention removal follow the basic removal relationships, and must recognize that not all of a downstream BMP’s volume can be used to treat the upstream flows. Our action: flow is allocated in proportion to the downstream BMP’s volume (the retention volume, or the permanent pool for wet detention) for each of the runoff volumes from multiple BMPs, and the overall effectiveness is calculated from that allocation.
- BMPs downstream of a wet pond. The first release of BMPTrains allowed none, when wet ponds were commonly the last BMP before discharge; the 2020 version allowed one. Our action: BMPFast allows many BMPs downstream of a wet pond, and testing compares hand calculations with the software’s results and the data it used.
- CN no greater than 95 for ROC. The Appendix N ROC tables were developed with an initial abstraction of 0.10 inches of rain, which corresponds to a maximum CN of 95 (not the 98 used for flood control). Our action: in testing, the input CN must reflect how ROC was calculated; the CN for a catchment cannot be greater than 95.
- Infiltration rates. Rates used in flood-control modeling should not be used for recovery when calculating annual pollutant loadings. Annual loadings use sustainable porosity and limiting infiltration rates, rather than the rates common in flood control, which apply to soils not subjected to solids loading from stormwater runoff. Our action: continue promoting and using the guidance in the Handbook and sustainable values.
- Harvesting. BMPTrains assumed harvesting treats a maximum annual runoff (ROC = 0.8) regardless of the catchment’s pervious and DCIA runoff conditions. Our action: the Handbook now requires ROC to be calculated, and it must be used consistently for BMP performance as well as runoff generation.
- A wet pond downstream of retention. Equation 9.5 recognizes that “the calculated overall efficiency of the treatment train must account for the reduced loading or concentrations that are available for removal by the subsequent downstream treatment device” (Handbook, page 9.6). BMPTrains assumed a constant decrease in the wet pond’s effectiveness regardless of the size of the retention upstream. Our action: calculations were developed for wet pond effectiveness as a function of the upstream retention treatment depth (cumulative inches of capture) and are in BMPFast. A spreadsheet is used for comparison with BMPFast’s results.
- Complex catchment configurations. BMPTrains was limited to configurations with only one upstream “strand”, and was not fully tested for BMPs in catchments interwoven in many different configurations. Our action: BMPFast was programmed to handle many strands in complex configurations, and extensive testing of various configurations is done and continues.
- Every addition is re-tested. Each time an addition is made to BMPFast, the test projects from the BMPTrains Manual, and any other test projects that may be affected, must be checked. At least 50 test projects are used.
- Retention that recovers in less than 72 hours. Average annual removal for retention systems that recover the design volume in less than 72 hours (vaults and exfiltration pipes) was programmed and tested. The equations are also presented in the BMPFast learning material (bmpfast.com/courses/bmpfast-certification-course/).
Summary remarks
- Initial testing compared results with commonly used existing software, with the input data modified to meet the Handbook. Some projects in the BMPTrains Manual were used for initial testing.
- Other projects reflecting new information in the Handbook were also tested. We viewed the calculations performed by BMPFast and compared them with “hand” calculations and a general understanding of the expected results.
- The background and equations used in the software are presented in the learning modules at bmpfast.com (bmpfast.com/courses/bmpfast-certification-course/), with lists of publications.
- Since July 1, 2024, over 140 projects have been received for comment; since July 2026, an average of 3 projects a week have been reviewed. Many have been used to test BMPFast, and all were for water management district (WMD) approval. All projects used for testing must be consistent with the Handbook and with the policies of the WMDs and other permit reviewers, so the software is flexible in the selection and acceptance of BMPs.
- We release parts of the software code for review by competent programmers, and appreciate those reviews. The code is kept in a secure repository with Microsoft (GitHub); releasing it for review there makes testing more complete while protecting the existing code.
- For review as well as testing, the results of the equations and the input data used can be displayed in the software. Much of this is already in the software’s reports and on the Equations page of each treatment worksheet. We welcome other suggestions.
- AI review and the help-page review process. AI tools are used as reviewers, never as decision-makers, in two ways:
- Calculation code. The code is reviewed for correctness only — lookup and interpolation, the efficiency tables, the special cases such as BMPs in series. Each finding is confirmed by running the calculation, not only by reading the code, and is tracked until it is fixed.
- Help pages against the program. Every window in BMPFast has a help page. Each page (more than 30) is compared, word by word, against what the program actually does: every input, label, unit, limit, default, button and result. Each disagreement or unclear point becomes a numbered question on that page (166 so far).
- Decision. The engineer responsible decides it, and the decision is recorded beside the question — a change to the program, a change to the help, a hand calculation, research, or no change with the reason.
- Change and test. A program change is built with its automated test where one applies, its screen is reviewed for clarity (a cognitive walkthrough and a heuristic evaluation), and the help draft is updated to match.
- Hand test. Anything the automated tests cannot show — a value reaching the calculation, a report reading correctly, a warning appearing, a worked example matching a hand calculation — becomes a hand-test item.
- Tracking. Every item — question, decision, hand test, screen capture, research — is kept on one to-do list by page and type, and in the development plan. An item is not complete until Dr. Ron Eaglin has verified it; passing tests alone do not close it.
- Outside review. The questions, decisions and proposed help pages are published to reviewers, who can send their own findings on any item, verified or open; those findings go back into the same process.
- We value professional input, and will continue to produce software of benefit to the profession for calculating annual water quality loadings of TP and TN when stormwater BMPs are used.
Item resolution and details are available to permitting agencies upon request — the question, the decision, the change made, and how it was verified. Email info@bmpfast.com.