On December 22, 2021, FDA published a request for comments on the newly issued draft guidance for industry entitled “Validation and Verification of Analytical Testing Methods Used for Tobacco Products” in the Federal Register (86 FR 72603). The recommendations provided address analytical testing of tobacco product constituents, ingredients, additives and stability testing of finished product. This guidance is to provide information and recommendations on how tobacco product manufacturers can produce validation and verification data for analytical methods used to support regulatory submissions for premarket approval. When finalized this draft guidance will represent the current thinking of the agency.
We applaud the agency for its decision to provide guidance to laboratories and other stakeholders to inform method validation and verification of analytical methods. This will aid in the standardization of assays with established accuracy, precision, repeatability, and reproducibility, resulting in the generation of data that can be used to make product comparisons and regulatory decision making.
Enthalpy Analytical submitted the following general comments listed below to the agency with the goal of providing clarity and strengthening the draft guidance document. Read our full comments, including an additional 18 specific comments to Docket No. FDA-2021-D-0756.
To hear more, join us at the Tobacco Merchants Association (TMA) annual meeting on March 29th, 2022 in Leesburg, VA, for the panel discussion entitled, “Addressing Upcoming Guidance on Testing Methodologies”. Andrew Cheetham, DPhil, a Senior Scientist in our Method Development Group will be a panelist and will discuss our position this guidance.
Outlined below are the following general comments we provided to the agency on the draft guidance.
1. FDA should be consistent with previous Method Validation Guidelines
In the absence of guidance from FDA CTP, we and others have adhered to the ICH Q2 (R1) guidelines for our internal method development and validation processes. While we support and applaud the agency in its decision to address the quality and rigor of analytical test methods that provide data for regulatory decision making in relation to tobacco and nicotine products, we also note that there exists extensive previous FDA and ICH validation guidance for foods, drugs, biologics, veterinary products, and medical devices. Although the provision of guidelines that communicate the expectations of FDA CTP with regard to method validation would be helpful to the industry, that those same guidelines would have industry deviate significantly from existing practices introduces unnecessary confusion for those trying to satisfy the requirements of both regulatory and accreditation agencies. It would be more prudent for FDA CTP to rely on and be consistent with this vast, well-documented body of prior work. The current draft guidance does not reference any of these previous documents.
2. FDA should promote standardized test methods
Tobacco and nicotine product manufacturers are required to submit listings of Harmful and Potentially Harmful Constituents (HPHCs) in their products and, as applicable, their emissions. HPHC comparisons are made to other product categories as well as products within the same category. These data are also an integral part of SE, PMTA and MRTPA submissions. The data are used by the FDA to make regulatory decisions. The validity and variability of this data are critical in this expensive effort. Meaningful product comparisons require that standardized methods exist with documented accuracy, precision, repeatability, and reproducibility. To this end, CORESTA has been responsible for the creation of numerous standardized methods for the analysis of HPHCs in combustible and smokeless tobacco products, many of which have been adopted as ISO methods. The majority of Enthalpy’s own analytical methods for HPHC determination adhere to these same recommended or standard methods as we were participants in the studies that led to the creation of these CORESTA recommended methods (CRMs). However, with the increase in the number of alternative nicotine delivery products entering the market in the last 15 years (ENDS, heated tobacco products (HTP), modern oral products (MOP)), there is a greater need for leadership and guidance from FDA CTP as to establishing standardized test methods for product comparisons within and across product categories.
Sample generation is a critical factor in the development of standardized methods and represents a significant challenge for the newer aerosol-generating devices (both ENDS and HTP). The National Academies of Sciences, Engineering, and Medicine (NASEM) 2018 report on public health consequences of e-cigarettes recommended the development of “standardized puffing protocols and validated methods to produce aerosols and analyze target constituents in e-cigarettes.” The same NASEM report concludes, “Much of the research on e-cigarettes suffers from methodological flaws and many important areas have not yet been researched.” As with combustible tobacco products, standardized aerosol-generating regimens are critical for cross-product comparative purposes, and to date only one such method exists for ENDS products, first developed by CORESTA as CRM 81 and later adopted by ISO as ISO 20768. FDA’s guidance on PMTA submissions for ENDS products recommends that two aerosol generation regimens be used in testing, one under non-intense puffing and the other under intense puffing. However, the specifics of those regimens are left to the discretion of the product manufacturer and/or the testing facility. This lack of standardization results in differences in testing regimens between products that make meaningful comparisons difficult if not impossible.
By providing insight into where the agency believes public health interests would be best served, FDA CTP will allow collaborative bodies such as CORESTA to better focus on the development of standardized methods that will ultimately lead to better data for the decision-making process. As Oldham concluded in a 2014 article, “Until capable laboratories use standardized assays with established repeatability, reproducibility, and certified reference standards, the resulting HPHC data will be unreliable for product comparisons or other decision making in regulatory science.“ While this quote was made in reference to cigarette and smokeless tobacco products, it is now more relevant than ever to the newer product categories on the market given the wide variety of product types available and the challenges they present to analytical testing systems.
Specific areas and/or actions that FDA CTP should contribute to include:
- Finalization of the HPHC list for ENDS e-liquid and aerosol. Potential additions were posted for comment in 2019 but no further action has been apparent since then.
- Creation of HPHC lists for the newer-to-market products such as HTP and modern oral products. These products feature very different matrices relative to traditional tobacco products, likely contain or emit, respectively, far fewer HPHCs, and as such warrant individualized HPHC lists.
- Involvement in the creation of standardized methods for the newer-to-market products. FDA CTP should increase its involvement with collaborative bodies to develop standardized testing methods for existing and any newly added HPHCs. It should also encourage independent testing facilities to adopt these new standardized methods when generating data for FDA submissions. Such actions can only serve to increase FDA CTP’s confidence in the quality of the data it uses for regulatory decision making.
- Provide guidance and expectations for aerosol sample generation. The utility of the data FDA CTP receives for making cross-product analyses in aerosol-generating products is governed by how comparable the aerosol generation is between those products. It is in FDA CTP’s best interest to have greater involvement in developing standardized methods for aerosol generation under conditions that provide the most valuable information for assessing a product’s potential impact on public health.
- Provide guidance for the study of long-term method variation. For stability studies, where HPHCs are measured over time, it is important to understand any temporal variation that may occur in an analytical method so as not to convolve it with test sample variation.
3. FDA should identify suitable certified reference products
The provided guidance states that “a validation of the ASP should include comparative measurements of sample extracts/smoke from a certified reference product (e.g., 1R6F, CM8, CRP1, etc.) and of the tobacco product category that is the subject of the validation.” This recommendation is wholly valid and good practice, but it does not acknowledge that no such reference products exist for the newer-to market products. Furthermore, what would the reference product be for aerosol-generating products such as ENDS and HTP? The device itself, the nicotine-containing component (e-liquid or tobacco substrate), or a combination of both? To add further complexity, both systems are available in a number of types – tank-based (open, refillable), pod-based (closed, exchangeable), and disposable (closed, sealed) for ENDS, and electrically-heated (eHTP), aerosol-heated (aHTP), and carbon-heated (cHTP) for HTP. Should one device type for each product class be used as a comparator for all types, or should a reference product for each product class variant be developed? CORESTA recently published a collaborative study report on the evaluation of a reference device (Aspire NautilusTM tank with an EvolvTM power unit) for e-cigarette aerosol generation that it intends to use as a monitor device in future e-vapor studies. Whether this device becomes a reference product for wider use remains to be seen, but it represents a step in the right direction.
If reference products are to be created for the more newly developed nicotine product types, then their design, development, and adoption will require significant input from FDA CTP in order for them to be appropriate for comparative testing purposes. FDA must participate in this crucial work.
4. Delineation of Analytical Measurement Procedure and Analytical Sampling Procedure
One of the major differences that these draft guidelines contain is the separation of the method validation into two parts, “analytical measurement procedure” and “analytical sampling procedure,” which other guidance documents address as a single concept. For example, the International Conference on Harmonization method validation guidance, ICH Q2(R1), terms this the “Analytical Procedure.” This delineation is also in contrast to guidance issued by other FDA centers, such as FDA CDER and CBER and the FDA Foods Program Regulatory Science Steering Committee (RSSC), which represents multiple centers. Furthermore, the currently adopted standardized methods for tobacco product analysis (both CORESTA and ISO) do not separate these aspects during their own development and validation. It would, therefore, be helpful to know what FDA CTP’s rationale is behind this choice as its implementation would represent a significant departure from current practices, as well as an additional burden with regard to experiments and documentation. Given this, the value of this separation becomes questionable since the guidance provides recommendations and not requirements. In our opinion, the guidance document would be clearer and more streamlined without this distinction.
Regardless of whether the delineation between the analytical measurement and sampling procedures is warranted, its implementation within the document suffers since the format and layout introduce further confusion due to inconsistencies and a lack of clarity throughout. For example, the first clear description of the delineation between analytical and sampling procedures occurs in Section V, paragraph 1 (page 13), and some version of it would have been better placed in Section VI, paragraph 3 (page 5), when the separation of the two elements was first introduced. The second and third paragraphs on page 6 do attempt to delineate the two, but some confusion is introduced by the use of the term “sample stock solution,” which is not defined in Section III (only “stock solution” is defined as referring to “a concentrated solution that may be diluted…for use in testing”). Under the definitions for “Analytical Procedure” and “Analytical Sampling Procedure,” the term “homogeneous sample” is mentioned alongside “stock solution,” with the inference being that this is the product sample that will be tested since step-by-step instructions are provided for its preparation. However, this term’s own definition could contradict its usage in this context since it is defined as a “portion of the material under test that is internally consistent and representative of the bulk of the material.” This could be taken to mean a sample extract that has been prepared through all the required sampling steps or simply a sample that has been prepared to undergo subsequent extraction and testing (i.e., compositing of material from multiple product cans to create a bulk sample, grinding of tobacco, etc.). Whether the final guidance maintains the delineation or not, there does need to be greater consistency in the terms and definitions used throughout so as not to add confusion and ambiguity.
5. Total Error Probability and Measurement Uncertainty
We find the inclusion of “total error probability” to be an interesting concept and certainly feel that it could present a viable alternative to the separate consideration of accuracy and precision for determining method suitability. It is certainly straightforward and simple to apply. However, we feel the term “total error probability” is somewhat of a misnomer as it implies the result is the probability that there is an error present. As defined both in the guidance and the source material, the probability being determined is of an assay value of 100% providing a passing result within defined acceptance criteria. As such, the “total error probability” actually provides an indication of how reliable the method is, and it is inversely proportional to the error present.
In the use of “total error probability,” the guidelines group accuracy and intermediate precision together to determine method suitability. ISO 17025 requires that accuracy and intermediate precision be determined separately along with the use of Measurement Uncertainty to capture determinate and indeterminate errors. No mention of Measurement Uncertainty is made within the guidance, though there is a recommendation that tobacco product manufacturers consider using accredited testing facilities. Consideration of the ISO 17025 requirements by which accredited testing facilities must abide would be welcomed.
Many of our method validations are based on the use of certified reference tobacco products (University of Kentucky, CORESTA) for which known (or accepted) values of many analytes have been determined. How does the use of these products for intermediate precision determination fit with total error probability since accuracy in this instance would be based on the closeness of the result to the established values rather than from spiking studies?
 International Conference on Harmonisation (2005), Validation of Analytical Procedures: Text and Methodology Q2(R1). (link)
 (a) FDA Center for Drug Evaluation and Research (CDER) and Center for Biologic Evaluation and Research (CBER), Guidance for Industry – Analytical Procedures and Methods Validation for Drugs and Biologics (July, 2015) (link); (b) FDA Office of Regulatory Affairs (ORA) Laboratory Procedure, ORA-LAB.5.4.5, APPENDIX 1-ORA Validation and Guidance for Human Drug Analytical Methods (August, 2014) (link); (c) FDA CDER and Center for Veterinary Medicine (CVM), Guidance for Industry-Bioanalytical Method Validation (May, 2018) (link); (d) FDA Office of Foods, Guidelines for the Validation of Chemical Methods for the FDA Foods Program, 3rd ed. (October, 2019) (link); (e) FDA CVM Guidance for Industry-Validation of Analytical Procedures: Methodology (July, 1999) (link).
 CORESTA CRM & ISO Correspondence List (Feb 2022) (link)
 National Academies of Sciences, Engineering, and Medicine 2018. Public Health Consequences of E-Cigarettes. Washington, DC: The National Academies Press. (link)
 CORESTA Recommended Method No. 81 – Routine Analytical Machine for E-Cigarette Aerosol Generation and Collection – Definitions and Standard Conditions (June, 2015). (link)
 ISO 20768:2018 Vapour products — Routine analytical vaping machine — Definitions and standard conditions (link)
 FDA Center for Tobacco Products, Premarket Tobacco Product Applications for Electronic Nicotine Delivery Systems. Guidance for Industry (June, 2019). (link)
 Oldham et al., Insights from analysis for harmful and potentially harmful constituents (HPHCs) in tobacco products, Reg. Tox Pharm 70(1), 138-148, 2014. (link)
 84 Fed. Reg. 38,032 (August 5, 2019). (link)
 Morton, M.J.; Gardner, W.P.; Agnew-Heard K.A.; Miller, J.H., “Temporal variability of analytical testing for e-vapor products and impact on number of replicates”. Verbal presentation at the 72nd Tobacco Science Research Conference, September 16-19, 2018, Memphis, TN, 2018 (link)
 CORESTA HTP Task Force, Heated Tobacco Products (HTPs): Standardized Terminology and Recommendations for the Generation and Collection of Emissions (July, 2020). (link)
 CORESTA E-Vapour Subgroup, 2019 Collaborative Study: Reference Device for e-Cigarette Aerosol (January, 2021). (link)
 FDA Center for Drug Evaluation and Research (CDER) and Center for Biologic Evaluation and Research (CBER), Guidance for Industry – Analytical Procedures and Methods Validation for Drugs and Biologics (July, 2015). (link)
 FDA Office of Foods, Guidelines for the Validation of Chemical Methods for the FDA Foods Program, 3rd ed. (October, 2019). (link)
 Ermer, J. and P. W. Nethercote (2014). Method Validation in Pharmaceutical Analysis. Singapore, Wiley-VCH. (link)