December 31, 2018 | Download PDF
By: Dr. Francis McVeigh, Science Director and Lead for Virtual Health Support Office
We at TATRC take the Secretary of Defense’s question very seriously and have internalized the Secretary’s question in all that we do. Before proposing new efforts and/or evaluating ongoing work, we dive deeper and ask ourselves a more specific question, “How do TATRC’s knowledge and materiel products increase the Warfighters’ lethality, will, and readiness?”
Before giving examples of TATRC’s efforts, I want to cite a few definitions that will help explain the type of work that TATRC does.
‘Science’ can be defined in many ways, but I have chosen the following definitions of Science: a branch of knowledge or study dealing with a body of facts or truths systematically arranged and showing the operation of general laws. A methodology for increasing understanding.
‘Research’ - The Department of Education and Training defines research as follows: the creation of new knowledge and/or the use of existing knowledge in a new and creative way so as to generate new concepts, methodologies and understandings. This could include synthesis and analysis of previous research to the extent that it leads to new and creative outcomes.
‘Evaluation’ - 1) focuses on programs vs. populations, 2) improves vs. proves, 3) determines value vs. stays value-free and 4) happens in real time. In light of these 4 points, evaluations, when carried out properly, have great potential to be very relevant and useful for program-related decision-making.
‘Integration’ - bringing together the component sub-systems into one system (an aggregation of sub-systems cooperating so that the system is able to deliver the overarching functionality) and ensuring that the sub-systems function together as a system, and in information.
‘Test and Evaluation’ - Test & Evaluation (T&E) is the process by which a system or components are compared against requirements and specifications through testing. The results are evaluated to assess progress of design, performance, supportability, etc.
IAW T. Beney (2011), Research and evaluation are characterized by similar features that center on the shared objective of answering a question. However, it is important to distinguish between the two disciplines by explaining that the purpose of evaluation is essentially to improve the existing program for the target population, while research is intended to prove a theory or hypothesis. Although both use similar data collection and analysis methods, the two disciplines diverge again during use and dissemination. Research is intended to increase the body of knowledge on a particular issue; any subjective opinion limits the researcher’s credibility. On the other hand, evaluators must balance the need to remain objective and the expectation to make recommendations for stakeholders. Evaluators must determine what information is valuable, what method is best for data collection, how to analyze the data, and how to relay findings to stakeholders. This requires interpretation and a certain level of judgment by the evaluator that is absent from the role of the traditional researcher.
Daniel L. Stufflebeam, Ph.D., a noted evaluator, captured it succinctly: “The purpose of evaluation is to improve, not prove3.” In other words, research strives to establish that a particular factor caused a particular effect. For example, smoking causes lung cancer. The requirements to establish causation are very high. The goal of evaluation, however, is to help improve a particular program. In order to improve a program, program evaluations get down-to-earth. They examine all the pieces required for successful program outcomes, including the practical inner workings of the program such as program activities.
TATRC as a hybrid organization, uses various scientific approaches to address the Secretary of Defense’s question, such as, research, evaluation, and integration. This combination and merging of approaches brings strength and value to the Warfighters by analyzing the same and different challenges in a myriad of ways. This approach enhances the likelihood that the end products will be meaningful and most importantly, put into the Warfighters’ hands sooner.
Some of the completed, ongoing and future targeted TATRC outcomes that are aimed at increasing lethality and readiness are as follows:
TATRC uses many different scientific approaches to tackle the myriad of multi-domain operations’ complex challenges. Make no mistake -- TATRC’s intent is to produce knowledge and materiel products that contribute to making the Warfighters more lethal and ready.
This article was published in the July 2019 issue of the TATRC Times.