U.S. Department of Health and Human Services
HHS
National Center for Advancing Translational Sciences
NCATS
Assay Guidance Manual Webtools
or load a sample dataset
Input Dataset
Output Dataset
Statistics
MSRn Table
Bland-Altman Figure
Correlation Figure

Replicate-Experiment Tutorial

The Replicate-Experiment App is a tool for generating appropriate assay statistics and visualizations to easily evaluate and document assay reproducibility. This tool can also be used to validate minor changes to assay protocols. More information regarding the Replicate-Experiment Study, including how and when it should be performed, can be found in the Assay Validation Chapter Section 4 and MSR Chapter, Section 2 of the AGM.

In its current form, the application requires the input data file to be a .csv file. Additionally, the input data file has the following requirements.

Input File Requirements (Potency or Efficacy Data)

  1. The input data file must have at least three columns with the following column titles. These titles are case-sensitive and must be spelled exactly as seen below. The input data set can have more than three columns, but the following columns are REQUIRED :

    1. Sample
    2. Exp1
    3. Exp2

    Optionally, you can include an additional column(s) to your Input Dataset. Like before, the title of this column is case-sensitive and must be spelled exactly as seen below:

    1. toExclude: Logical values to highlight which rows to exclude
    2. asControl: Logical values to highlight which rows represent control trials
  2. For the required Sample column, the data type is not restricted. Your values can be numbers or strings. Please avoid using logical values here without any additional strings appended to them.

  3. The required columns Exp1 and Exp2 must only contain numerical values. This means that there can be no letters included in any of the data values. Additionally, please do not use any other special characters other than minus signs and decimals. If there are values of Not a number (NaN) in your dataset, set those to blank instead. Examples are included below of what does and does not constitute numerical values for the purposes of this application.

Accepted Unaccepted
1 3xc9*
3.12 '24'
-0.54 skip
(454)
  1. For the Optional toExclude and asControl columns, make sure that the values entered into these columns are logical. Specifically, for a given row, only write the words: TRUE or FALSE. It is also acceptable to not write anything at all. For example, if you are selecting samples to be excluded, write the word TRUE in the rows you want to exclude under the toExclude column. For all other rows in the toExclude column, keep them blank or write the word FALSE.
Note: Input Data Units

Please use whatever units you would like for the columns of Exp1 and Exp2 in the input dataset. The app will function as expected as long as units are consistent; the data displayed will always be in the same units as the input data. At present there is no plan to offer any “in-app” unit conversions of the data.

Exporting Tables

At the bottom of page, there is a large rectangular button that says Download Tables. This will download the Output Dataset Table and the Statistics Table.

Exporting Figures

At the top right of each figure, there are four icons that allow further interaction with the image. The leftmost icon (which looks like a camera), is the button to Download plot as a png. This is the current best method for downloading each individual image to your downloads folder. As a note, if you choose to interact with the image such as zooming and panning, the exported png will be of the same view as you seen after the manipulation..i.e. the same level of zoom and direction of panning.

Plotly Modebar

or load a sample dataset

Control Charting Tutorial

The Control Charting App is a tool for generating figures and statistics that assess assay performance across runs. The goal of control charting is to identify when and how your assay changes over time. More information regarding Control Charting, including how and when it should be performed, can be found in Production Monitoring Section and MSR Chapter, Section 2 of the AGM.

In its current form, the application requires that the input data file to be a .csv file. Additionally, the input data file has the following requirements.

Input File Requirements

  1. The input data file must have at least two columns with the following column titles. These titles are case-sensitive and must be spelled exactly as seen below. The input data set can have more than three columns, but the following columns are REQUIRED :

    1. Run
    2. Data

    Optionally, you can include an additional column(s) to your Input Dataset. Like before, the title of this column is case-sensitive and must be spelled exactly as seen below:

    1. Replicate_ID: Category of replicates for the data being
  2. For the required Run column, the data type is not restricted. Your values can be numbers or strings. Please avoid using logical values here without any additional strings appended to them.

  3. The required column Data must only contain numerical values. This means that there can be no letters included in any of the data values. Additionally, please do not use any other special characters other than minus signs and decimals. If there are values of Not a number (NaN) in your dataset, set those to blank instead. Examples are included below of what does and does not constitute numerical values for the purposes of this application.

Accepted Unaccepted
1 3xc9*
3.12 '24'
-0.54 skip
(454)
  1. For the optional Replicate_ID columns, please use only numerical values.
Note: Input Data Units

Please use whatever units you would like for the columns of Exp1 and Exp2 in the input dataset. The app will function as expected as long as units are consistent; the data displayed will always be in the same units as the input data. At present there is no plan to offer any “in-app” unit conversions of the data.

Exporting Tables

At the bottom of page, there is a large rectangular button that says Download Tables. This will download the Output Dataset Table and the Statistics Table.

Exporting Figures

At the top right of each figure, there are four icons that allow further interaction with the image. The leftmost icon (which looks like a camera), is the button to Download plot as a png. This is the current best method for downloading each individual image to your downloads folder. As a note, if you choose to interact with the image such as zooming and panning, the exported png will be of the same view as you seen after the manipulation..i.e. the same level of zoom and direction of panning.

Plotly Modebar

Privacy Notice Statement

This is a government resource provided by the National Institutes of Health (NIH) National Center for Advancing Translational Sciences (NCATS). Protecting your privacy is important to us. NCATS complies with the requirement for privacy and security established by the Office of Management and Budget (OMB), the Department of Health and Human Services (HHS), and the National Institutes of Health (NIH). This page outlines our privacy policy as it applies to our site, NCATS Assays Guidance Manual (NCATS AGM Webtools), as well as third-party sites and other publicly available NCATS resources or applications.

NCATS AGM Webtools will not collect any personal information about you unless you choose to explicitly provide it to us. We do, however, collect some data about your visit to our website to help us better understand the public use of the site and to make it more useful to visitors.

Types of Information Collected

When you browse through any website, certain information about your visit can be collected. NCATS automatically collects and temporarily stores the following type of information about your visit:

  • Domain from which you access the Internet.
  • IP address (an IP address is a number automatically assigned to a computer when surfing the Web).
  • Operating system and information about the browser used when visiting the site.
  • Date and time of your visit.
  • Pages you visited.
  • Address of the website that connected you to NCATS (such as google.com or bing.com).

NCATS uses this information to measure the number of visitors to our site and its various sections and to help make our site more useful to visitors.

How Information is Collected by NCATS

This website uses Google Analytics measurement software to collect the information in the bulleted list under Types of Information Collected. Google Analytics collects information automatically and continuously. No PII is collected. NCATS staff and contractors conduct analyses and report on the aggregated data from Google Analytics. The reports are only available to this website’s managers and other designated staff who require this information to perform their duties. NCATS retains the data from Google Analytics as long as needed to support the mission of this website.

How NCATS Uses Cookies

The Office of Management and Budget (OMB) memo M-10-22, Guidance for Online Use of Web Measurement and Customization Technologies allows federal agencies to use session and persistent cookies.

When you visit any website, its server may generate a piece of text known as a “cookie” to place on your computer. The cookie allows the server to “remember” specific information about your visit while you are connected.

The cookie makes it easier for you to use the dynamic features of Web pages. Cookies from NCATS Web pages only collect information about your browser’s visit to the site; they do not collect personal information about you.

There are two types of cookies, single-session (temporary) and multi-session (persistent). Session cookies last only as long as your Web browser is open. Once you close your browser, the cookie disappears. Persistent cookies are stored on your computer for longer periods.

Session Cookies: We use session cookies for technical purposes, such as to enable better navigation through our site. These cookies let our server know that you are continuing a visit to our site. The OMB memo 10-22 guidance defines use of session cookies as “Usage Tier 1—Single Session.” The policy says, “This tier encompasses any use of single session Web measurement and customization technologies.”

Persistent Cookies: We use persistent cookies to enable Google Analytics to differentiate between new and returning ncats.nih.gov visitors. Persistent cookies remain on your computer between visits to this website until they expire. The OMB memo 10-22 guidance defines use of persistent cookies as “Usage Tier 2—Multi-session without Personally Identifiable Information (PII).” The policy says, “This tier encompasses any use of multi-session Web measurement and customization technologies when no PII is collected.”

Opt Out or Disable Cookies: If you do not wish to have session or persistent cookies placed on your computer, you can disable them using your Web browser. If you opt out of cookies, you will still have access to all information and resources at ncats.nih.gov. Find instructions for disabling or opting out of cookies in the most popular browsers. Note: By following the instructions to opt out of cookies, you will disable cookies from all sources, not just those from ncats.nih.gov.

By using our websites, you consent to uploading experimental data and using NCATS AGM web tools to perform statistical analysis to generate data visualizations. NCATS AGM does not store or maintain experimental data. All information is deleted at the termination of a user session.

If we decide to change our privacy policy, we will post those changes on this page so that you are always aware of what information we collect, how we use it, and under what circumstances we disclose it.

How is Information Protected

You do not have to give us personal information to visit the NCATS AGM Webtools website or browse information hosted on this site. However, should you choose to provide your information that identifies you through an (email message, form, survey, etc.), we will maintain the information you provide only as long as needed to respond to you.

Disclosure

NCATS does not disclose, give, sell, or transfer any personal information about our visitors unless required for law enforcement or by statute. For further information about the NIH privacy policy, please contact the NIH Office of the Senior Official for Privacy or visit the NIH Privacy Program.

Privacy Office

NIH Office of the Senior Official for Privacy (OSOP)
National Institutes of Health, Office of Management Assessment
6011 Executive Blvd., Suite 601, MSC 7669
Rockville, MD 20852

Security/Site Security

The U.S. federal government maintains this site. It is protected by various provisions of Title 18, U.S. Code. Violations of Title 18 are subject to criminal prosecution in federal court.

For site security purposes and to ensure that this service remains available to all users, we employ software programs to monitor traffic to identify unauthorized attempts to upload, change information, or otherwise cause damage. In the event of authorized law enforcement investigations and pursuant to any required legal process, information from these sources may be used to help identify an individual.

Use of Third-Party Website and Applications & non-Federal sites

Third-party websites and applications are web-based technologies that are not exclusively operated or controlled by NCATS or NIH. NCATS may use these technologies and social media tools to communicate and interact with the public. The use of these applications could cause personally identifiable information (PII) to become available or accessible.

When a visitor accesses or follows an embedded link of a third-party site, you will be subject to the destination site’s privacy policy. Any information you choose to provide to third party organizations through registration, posting, blogging, or other interactions is voluntarily contributed and is not collected by NCATS. Linking to a non-federal site does not constitute an endorsement by NCATS or any of its employees of the sponsors or the information and products presented on the site.

Endorsement Disclaimer

NCATS does not endorse or recommend any specific commercial products, process, service, manufacturer, or company on its website. Further, the mention of products does not constitute HHS endorsement or recommendation by any HHS employees.

Links with this icon external link icon indicate that you are leaving NCATS (website). The link will lead to a non-federal site but provide additional information that is consistent with the intended purpose of a federal site.

Information Disclaimer

NCATS websites may provide links to other non-Federal Internet sites for informational purposes and the convenience of its users. The National Institutes of Health (NIH), which includes NCATS, cannot attest to the accuracy of the information on a non-federal site. The information provided on the (NCATS website) is intended to be general summary information for the public. It is not the intent to take the place of either the written law or regulations.

When users select a link to an external website, they leave the NCATS website and are subject to the destination site’s privacy and security policies of the owners/sponsors of the external site.

Pop-Up Advertisement Disclaimer

When visiting the NCATS (‘website’), your Web browser may produce pop-up advertisements. These advertisements most likely were produced by other websites you visited or third-party software installed on your computer. NCATS and the NIH do not endorse or recommend products or services for which you may view a pop-up advertisement on your computer screen while visiting this site.

Contact the NCATS AGM Webtool

If you have questions or concerns about the NCATS Assays Guidance Manual tool, please contact NCATSAGMWebToolSupport@mail.nih.gov.

Feedback Form

We would love to hear your feedback on this app. Please click the button below to go to the feedback form.

Submit Webtool Feedback
Authors Page

Development Team

Data Scientists

Dante Smith
Dante Smith
NIH NCATS/NCI
Nina Mezu
Nina Mezu
NIH NCATS
Andrew Weisman
Andrew Weisman
NIH NCATS/NCI

Assay Guidance Experts

Jeff Weidner
Jeffrey R. Weidner
Founder, QualSci Consulting, LLC
Philip Iversen
Philip Iversen
Associate Professor of Mathematics, Luther College
Jason Manro
Jason Manro
Eli Lilly and Company
Mary Jo Wildey
Mary Jo Wildey
Merck & Co., Inc. (Retired)

Project Management/Leadership

Sarine Markossian
Sarine Markossian
Editor-in-Chief, Assay Guidance Manual(AGM)
Lead, AGM Translational Science Resources Program, NCATS
Kelli Wilson
Kelli Wilson
Director, Compound Management Group, NCATS
Hannah Baskir
Hannah Baskir
Scientific Program Manager
AGM Translational Science Resources Program, NCATS