A New Framework for Evaluating Cannabinoid Products for Safety, Quality, and Efficacy

CBD Oracle interviewed 22 experts and two government agencies to answer a simple question: how can you choose safe, effective, and reliable hemp products?

Written by

Lee Johnson

Lee Johnson is the senior editor at CBD Oracle, and has been covering science, vaping and cannabis for over 10 years. He has a MS in Theoretical Physics from Uppsala...

CBD Oracle's Editorial Process
cannabinoid tincture bottle cannabis leaf
Illustration: Layla Selestrini / CBD Oracle

We love hemp, but the industry has a long way to go. There’s widespread deceptive labeling, undeclared (and sometimes even unknown) ingredients, a shameful lack of age-gating for online purchases, products evading safety testing, a culture of silent complicity among testing labs, high-THC products that don’t mention that they’re intoxicating, faked lab reports and a dizzying array of products that look like candy from the outside but get you higher than any edible you’d pick up from a cannabis dispensary.

The FDA has dragged its feet on regulation and customers are left to navigate this confusing and sometimes dangerous gauntlet for themselves.

But hemp and CBD still hold a lot of value, offering medical benefits for a wide range of conditions and allowing consumers to access intoxicating products even if their state still prohibits cannabis.

Despite many problems, there are tons of companies out there that do get things right and take the necessary steps to protect consumers.

So how can customers be sure that what they’re buying is safe, effective and reliable? Are there any benchmarks consumers and the industry themselves can use to make sure their hemp products meet high standards?

We’ve spoken to industry experts and reviewed existing standards to come up with some objective guidelines.


The framework is summarized below. You can view the full report here (PDF).


What We Did: Expert Interviews and Research on Existing Frameworks

Most hemp consumers know the basic advice for finding a good product. You check the Certificate of Analysis (COA), look for a few customer reviews and if everything checks out, you hit “Buy” and hope for the best.

But even though this is a great start, there are so many ways this can go wrong and many important details that it glosses over or outright misses.

To rectify this, we got input from 22 experts from the industry, testing labs, medical professionals, lawyers and researchers, along with two governmental agencies.

List of experts who were interviewed for this report

We asked them what they would look for in a hemp product if they were recommending them to a friend, family member or patient (for doctors with patient experience), and got insight from them on specific factors relevant to their experience.

We also reviewed existing standards and frameworks, including standards from the US Hemp Authority, ASTM’s cannabis standards, FLOW Criteria, USP cannabis standards, regulations from states such as California and Colorado, and more general frameworks like FDA Good Manufacturing Practices.

Additionally, we drew on our own experience and knowledge of the industry to center the whole discussion on the needs of consumers and ensure that the resulting framework was actionable and practically useful.  


Why We Did It: The Problems With the Hemp Market

The Patchwork of Regulations

The hemp market does not have consistent regulation.

Shawn Hauser, partner at Vicente LLP cannabis law firm, explained to us that, “The FDA’s failure to establish regulations (and its resulting 50 state patchwork of constantly changing laws) for finished hemp products remains the primary legal challenge for the hemp industry. After 5 years assessing potential regulations, FDA decided to defer to Congress to develop appropriate regulations, while it maintains jurisdiction.”

Some states have developed detailed rules for hemp products sold in-state (such as Tennessee), but most states still retain the bare minimum regulation and the FDA still has not taken any substantial steps to regulate the industry at the federal level.

In fact, as Shawn pointed out, they’ve deferred to Congress, who in turn issued a request for information on CBD regulations in summer 2023.  

Shawn continued, “For the hemp industry, this means 50 different state rules as to whether hemp products can be manufactured and sold, licensing, product potency, serving and container size, packaging/labeling, marketing, testing, and points of sale.”

This has ultimately led to the situation we have today. Hemp products are sold across the United States with only minimal oversight, usually in direct violation of the Food, Drug and Cosmetic Act and often without any legally-enforced age restrictions on sales.

The lack of top-down regulation means that even things that are ostensibly legal requirements – such as a limit of 0.3% delta-9 THC by dry weight – are not enforced and are rarely even checked in practice.

Bad Actors Operating in the Hemp Industry

This is all made worse by bad actors operating in the hemp industry.

The FDA, for example, regularly sends warning letters to companies making unapproved medical claims, which range from relatively minor (e.g. blood pressure) to obviously serious (e.g. cancer and cardiovascular disease). And even beyond this issue, the majority of hemp products are not accurately labeled, so you cannot be sure of the amount of CBD, THC, or anything else you’re actually getting.

Even worse, while CBD and delta-9 THC are the cannabinoids you might expect, there has been an explosion in the availability of alternative intoxicating cannabinoids, including delta-8 THC, delta-10 THC, hexahydrocannabinol (HHC), THCjd, THCV, and more. These are generally not very widely tested or well understood and are almost always accompanied by unintended byproducts that carry unknown risks.

Certificates of Analysis and Their Limitations

In large part through consumer pressure, it is now an accepted norm in the industry to have a Certificate of (lab) Analysis (COA) accompanying each hemp product.

However, this is the only assurance given to customers the majority of the time, and even this is not always reliable. In fact, in many cases, COAs shared by companies do not match the results of independent testing of the same products.

Dr. Abraham Benavides, an international medical cannabis coach, and consultant, told CBD Oracle that “we are in a confusing time when certain labs are not fulfilling their roles honestly and may be subject to corruption for favorable results – commonly called ‘lab shopping.’ This means that we can’t solely rely on COAs for selection, though its absence is still a ruling out factor.”

This leaves consumers looking for safe, effective, and reliable products in an extremely difficult situation.

There is no “bare minimum” regulatory standard they can expect products to meet, even lab test reports are not necessarily reliable and it’s unlikely any product will actually contain the amount of CBD or other cannabinoids promised on the label.

There is no clear, detailed framework for choosing safe, effective and reliable hemp products.


How This Affects Consumers

With all of this, consumers who are hoping to use CBD or another cannabinoid to manage health issues or for any other reason are the ones paying the price.

  • Inaccurate labeling: Getting the correct dosage is vital to managing health issues, so inaccurately labeled products can lead to consumers only getting an inadequate dose. This could possibly lead to a false conclusion that CBD – or whichever cannabinoid – is not effective for their condition.
  • Deceptive marketing: If a product is advertised as effective for a certain health condition, consumers with that condition are more likely to buy it. However, if this claim is made inaccurately, the consumer may choose an ineffective product instead of a more traditional treatment that would be effective.
  • Unsafe products: Novel cannabinoids are increasingly linked to reports of adverse events and calls to poison control centers, possibly due to widespread unintended byproducts. Even well-known cannabinoids are often sold in products without adequate safety testing, opening users up to risks.
  • Inconsistent products: Even if a product works the first time a consumer tries it, if the next batch they buy has much less of the active ingredient, they may falsely conclude the product does not work. Similarly, a THC product may be mildly intoxicating one time, but the next time the consumer purchases, it could be much stronger and lead to an unpleasant experience.

All of these things undermine consumer confidence in the industry, and ultimately lead to consumers turning away from products which could actually help them.


Our Proposal: Cannabinoid Product Evaluation Criteria

To address these issues, we looked into existing standards for cannabis and hemp products and interviewed 22 experts about the issues, as well as getting comments from the FDA and the California Department of Public Health.

The aim of these criteria is to offer a set of guidelines that are as objective as possible and that can be checked by a third party.

The criteria are geared towards four key goals:

  • Safety: Products should have a low potential to cause harm.
  • Efficacy: Products should contribute notably to healing, or at least contain a potentially useful amount of specific cannabinoids.
  • Reliability: Products should offer consistent results every time, particularly with regard to safety and efficacy.  
  • Trust: Products should be made by a company that is transparent, responsible, and reputable.

While most of these criteria can be checked by a third party, in some cases companies will not have all of the information available publicly. However, companies should be able to confirm any remaining points if contacted by email.

It’s also important to note that just because a product doesn’t meet each of these criteria doesn’t mean that it’s bad. It merely suggests that there are areas of uncertainty or some negative points about the product.


CBD Oracle’s Cannabinoid Product Evaluation Criteria

Golden boxes illustrating quality hemp products
Illustration: Layla Selestrini / CBD Oracle

A summary of the framework is provided below. The full report can be accessed here: A Comprehensive Framework for Evaluating Cannabinoid Products

1. The Plant’s Origin and Cultivation Method

  • The Crop Is Grown in the United States
  • The Hemp Is Grown in States With Ideal Climates and Robust Hemp Programs
  • The Hemp Is Grown Outdoors Under Natural Sunlight
  • The Hemp Is Grown Using Organic Farming Methods
  • The Hemp Is Sourced From a Single Origin
  • The Product Is Extracted From a Single Strain

2. Extraction

  • The Extract Is Made Using a Clean Method That Removes Contaminants and Preserves Beneficial Compounds
  • The Cannabinoids Are Naturally Extracted, Not Chemically Converted
  • The Product Contains a Full-Spectrum or Broad-Spectrum of Cannabinoids
  • If the Product Is an Isolate, It Contains 99% or Higher of a Single Cannabinoid
  • The Extract Is Rich in Natural Terpenes
  • The Extract Is Flower-Derived

3. Third-Party Lab Testing

  • Full-Panel Lab Test Results Are Publicly Available for Both Raw Materials and Finished Products
  • The Product Is Tested by an ISO/IEC 17025 Accredited Lab With a Credible Reputation
  • A Certificate of Analysis Is Provided for Each Batch, Lot, and Product Variation
  • Potency Test Results Are Within 10% of the Advertised Level, or 20% for Low-Dose Products
  • Lab Results Are Consistent Across All Batches
  • Lab Results Confirm the Product Is Free From Contaminants and Residual Solvents
  • The Lab Results Confirm Multiple Cannabinoids and Terpenes If the Product Claims to Be Full-Spectrum or Broad-Spectrum
  • The Lab Result Includes the Terpene Profile of the Product
  • The Lab Report Is Less Than 12 Months Old
  • The Lab Report Includes a QR Code for Easy Verification of Results
  • The Lab Report Includes a Chromatogram to Confirm Accurate Product Analysis 
  • Lab Testing for Converted Cannabinoids Distinguishes Between Delta-8 THC and Common Byproducts  

4. Quality Control

  • The Hemp Is USDA-Certified Organic
  • The Manufacturer Is Certified for Good Manufacturing Practices (GMP)
  • The Product Is Made in an FDA-Registered Facility
  • The Product Has NSF Certification, If Applicable 
  • The Company Can Provide a Chain of Custody Document for the Hemp
  • The Company Oversees the Entire Production Process (Seed-to-Sale) and Is Not a White-Label Brand

5. Ingredients and Additives

  • The Product Is Made With Clean Ingredients and Contains No Harmful Additives or Synthetic Compounds 
  • The Product Avoids Unregulated Supplements Like Melatonin or Valerian Root 
  • Vape Products Are Free From Unlabeled Cutting Agents and Heavy Metals, and Use a Base of Propylene Glycol and/or Vegetable Glycerin 

6. Packaging and Labeling

  • The Product Is Packaged in a Child-Resistant Container
  • The Label Includes a QR Code to Verify Lab Results 
  • The Label Displays the Packaging and Expiration Date 
  • The Label Provides Accurate Dosing Instructions 
  • The Product Complies With Existing Labeling Regulations  
  • The Label Includes a Batch and Lot Number for Traceability  
  • The Label Includes Appropriate Health Warnings 
  • The Label Lists Nutritional and Content Information
  • The Product Label Clearly Lists All Active and Inactive Ingredients

7. Reputation, Transparency, and Marketing

  • The Company Avoids Making Unapproved Medical Claims on Its Website or Labels 
  • The Company Avoids Marketing That Appeals to Teens, Such as Cartoon Images or Candy-Like Designs 
  • The Company Requires Age Verification and Delivery Signatures for Intoxicating Products 
  • The Company Shares Clear Information About Its Founders and Team 
  • The Company Provides Transparent Details About Sourcing, Production, and Lab Testing
  • The Company Collaborates With Experts in Medical, Scientific, and Legal Fields 
  • The Company Does Not Remove Negative Customer Reviews From Its Website
  • The Company Has Positive Reviews on Third-Party Platforms and Actively Engages With Customers
  • The Company Is a Member of Industry Organizations and Supports Advocacy Efforts
  • The Company Offers a Money-Back Guarantee
  • The Company Has Not Received FDA or FTC Warnings in the Past 12 Months
  • The Company Has Been in Business for Over Three Years
  • The Company Provides Prompt and Knowledgeable Customer Service

Why This Is Useful

This framework offers a clear, objective way to evaluate cannabinoid products, going far beyond unclear advice to just “do your research” or check lab reports.

Specifically, there are three main ways the criteria can help to improve the current situation for the industry and consumers alike.

Educating and Empowering Consumers and Healthcare Professionals

The biggest benefit of the evaluation criteria is to help individual consumers and healthcare professionals to choose high-quality and reliable products. Sticking to the recommendations will ensure that products are as safe as possible, that they offer the cannabinoids they claim to and that they are as consistent as possible.  

Encouraging Manufacturers to Self-Regulate to a Higher Standard

Establishing a clear set of guidelines also gives the industry a set of criteria to consider for their products. For example, while a manufacturer may get each product lab tested, the criteria make it clear that each batch/lot should be tested for each SKU.

The knock-on effect is that fewer problems would be created by hemp products, and therefore politicians will have less justification for harshly restricting the industry.  

Informing Policymakers

Much of the discussion around hemp is uninformed and relies on soundbites (e.g. referring to the industry as a “wild west”). The criteria give a solid checklist for what consumers expect from the industry and what the best companies in the space are already doing.

The criteria are solely concerned with the safety and reliability of products, offering judgment-free insight into the most pressing issues for consumers and the industry. It could even serve as a guide for how to regulate hemp, along with other existing standards.


Conclusion: Where Do We Go From Here?

So how do we turn this from a simple proposal into something that has a real impact on the industry?

The most important thing for consumers is to expect this level of quality from any hemp product you buy. Just like continued pressure from consumers has made COAs commonplace throughout the industry – even when not required by law – making sensible purchasing decisions encourages other manufacturers to bring their products up to this standard.

Any improvements from these criteria will not materialize immediately, but setting a benchmark and sticking to it gives companies something to shoot for.

Until we get substantial, top-down regulations for the industry, there will always be companies that opt to do the bare minimum to make a quick buck. But anyone who wants to get consistent business from dedicated consumers will have to rise up to the standards we expect from them.


RELATED:

Advertisement for THC vaporizer