- Journal List
- HHS Author Manuscripts
- PMC8686184
As a library, NLM provides access to scientific literature. Inclusion in an NLM database does not imply endorsem*nt of, or agreement with, the contents by NLM or the National Institutes of Health.
Learn more: PMC Disclaimer | PMC Copyright Notice
J Dev Behav Pediatr. Author manuscript; available in PMC 2021 Dec 20.
Published in final edited form as:
J Dev Behav Pediatr. 2021 Jan 1; 42(1): 55–60.
PMCID: PMC8686184
NIHMSID: NIHMS1754463
PMID: 32881771
Lauren H. Hampton, PhD, BCBA,1 Megan Y. Roberts, PhD, CCC-SLP,2 Erica Anderson, PhD,2 Amanda N. Hobson, B.S.,2 Aaron J. Kaat, PhD,2 Somer L. Bishop, PhD,3 Sheila Krogh-Jespersen, PhD,2 Lauren S. Wakschlag, PhD,2 and Katherine B. Bevans, PhD4
Author information Copyright and License information PMC Disclaimer
The publisher's final edited version of this article is available at J Dev Behav Pediatr
Abstract
Approximately 50% of children with autism exhibit severe tantrums, defiance, and/or aggression. We propose that the Disruptive Behavior Diagnostic Observation Schedule (DB-DOS), a standardized clinical observation modeled after, and complementary to, the Autism Diagnostic Observation Schedule (ADOS) could enhance earlier identification of DB in autism populations and inform treatment planning. We adapted the DB-DOS for children with autism based on expert input and preliminary feasibility testing to accommodate varying cognitive and social-communication capacities and increase the likelihood of observing DB in this population. Thereafter, we concurrently administered the modified DB-DOS and the ADOS to 12 children with autism aged 36–50 months. Overall, children exhibited greater DB, especially behavioral regulation challenges during the DB-DOS than during the ADOS. The use of a developmentally-sensitive standardized observation tool that presses for DB to complement standardized observations such as the ADOS shows promise for enabling more precise research on targeted DB interventions. Such a tool holds promise as a reliable and efficient method of identifying co-morbid DB disorders in the autism population.
Keywords: Autism, Disruptive Behavior, Assessment, Challenging Behavior
About 50% of children with autism exhibit severe tantrums, defiance, and/or aggression.1 In addition, challenging behaviors are significantly greater for young children with autism than for other at-risk populations.2 Disruptive behavior (DB) in early childhood encompasses severe forms of the normative misbehaviors of this developmental period including temper tantrums, aggression and noncompliance.3 Disruptive behavior (DB) limits opportunities for inclusion, and thus exerts a negative cascading impact by reducing exposure to age-appropriate models and participation in community settings. Psychopharmacologic and behavior interventions are commonly used to treat DB symptoms in young children with autism, yet developmentally-tuned assessments for determining the need for and effectiveness of such treatments are lacking.
A recent systematic review found that none of the DB measures used in autism research have adequate evidence of autism-specific content validity.4 Many measures do not distinguish intentional aggressive acts from impulsive, unintended reactions to provocation or interruption of a perseverative activity. Informant-report items such as “calling others names” may be irrelevant for children with language delays, and “doesn’t answer when people talk to him/her” may confound noncompliance with social communication and/or receptive language deficits.
Although DB may be observed during administration of the Autism Diagnostic Observation Schedule5 (ADOS), this tool’s emphasis on scaffolding to optimize child engagement may reduce the likelihood of eliciting DB except for in cases of extreme dysregulation. Building on the ADOS paradigm, Wakschlag et al. developed the Disruptive Behavior Diagnostic Observation Schedule (DB-DOS), a standardized clinical observation paradigm that distinguishes young children’s normative misbehavior from clinically concerning DB.6,7 The ADOS and DB-DOS observational paradigms have opposite behavioral elicitation objectives. Whereas the ADOS scaffolds to optimize engagement, the DB-DOS deliberately presses for difficulties in regulating frustration and adapting behavior flexibly to environmental demands.
The DB-DOS assesses DB in parallel parent and clinician contexts. This has the dual objective of maximizing ecological validity by assessing behavior with a familiar adult while at the same time testing the limits of the child’s capacity to self-regulate in the absence of typical scaffolding. This provides a unique opportunity to evaluate the extent to which parental behavior may amplify or constrain dysregulated behavior as well as to observe child DB with a non-parental adult (which may serve as a proxy for behavior with teachers).8 In both contexts, the child is exposed to standard “do” and “don’t” compliance tasks designed to elicit mild frustration or disappointment (e.g., difficult puzzle, malfunctioning toy). Examiner responses to child behavior are structured along a hierarchy of increasing support to assess the minimal level of input needed for children to manage their behavior. Parent’s behavior is unscripted to enable observations of the child’s self-regulation and parents’ responses to child misbehavior in contexts that mirror daily life (thus, providing insights not only about child DB but parental behavior that may trigger or modulate it). Quality of misbehavior are scored along a clinical continuum in both the examiner and parent contexts. Disruptive behaviors are scored across two domains: Problems in Behavioral Regulation and Anger Modulation.9 Behavior Regulation reflects the extent to which children can regulate response to environmental demands across the compliance/noncompliance domain including quality of noncompliance such as overt defiance, ignoring, provocativeness and stubborn, inflexible behavior. The Anger Modulation reflects (dys)regulation of irritable/angry affect elicited by frustration and limits includes the intensity, predominance, ease of elicitation, recovery and coping, and the escalation and de-escalation.
The DB-DOS has strong inter-rater and test-retest reliability and concurrent and predictive validity for psychopathology among children without autism, and has recently been adapted for use in infants/toddlers which is important for accommodating the full spectrum of functioning in young children with ASD.6,7,9–12 Variation in DB-DOS scores across parent and examiner contexts detect gender differences,13 and converges with concordance/discordance across parent and teacher-reported DB and has demonstrated incremental utility above and beyond DSM symptoms.7,8 However, for the DB-DOS to be useful for children with autism, it must elicit and capture DB common in children with autism (differentiating autism related deficits such as lack of comprehension or perseveration from clinically significant DB) and accommodate for core and related ASD symptoms. Specifically, DB-DOS modifications are needed to (1) differentiate deliberate ignoring from failure to understand directives; (2) detect nonverbal expressions of non-compliance and bossy/argumentative behavior; (3) elicit responses to withholding or removing access to restricted interests or preferred objects vs general environmental demands; while (4) maximizing opportunities to observe parents’ behavior management strategies and the opportunity to contrast the child’s behavior with the parent or with a “semi-standardized” non parental adult to inform tailored treatment planning.
This paper describes the process of tailoring the DB-DOS for use with children with autism and presents a proof-of-concept demonstration of the tool’s feasibility and utility for this population and pilot adaptations specific for the autism population. We were keenly aware that assessment for autism is time-intensive and specialized, thus principles of pragmatic assessment were fundamental to this process.14 In an effort to optimize measurement sensitivity while minimizing assessment burden, we contrasted the patterns of DB observed using: (a) the standard DB code on the ADOS; (2) application of DB-DOS codes to behavior during the ADOS; and (3) actual administration and coding of the DB-DOS.
METHODS
Pilot administration
Initially, we administered the standard DB-DOS protocol, without any modifications, to 3 children previously diagnosed with autism (boys ages 5–7). These children were specifically selected to represent a wide range of parent concern about disruptive behavior, cognition (one low, one moderate, one high), and functional language (one minimally verbal, one moderate, one average verbal ability). Our team of experts in autism, the ADOS, and standardized observation measures, including the developer of the DB-DOS, made several modifications to the DB-DOS protocol to improve its utility for children with autism (Table 1). Next, we concurrently administered the adapted DB-DOS protocol and the ADOS to 12 new participants with autism.
Table 1.
DB-DOS tasks and presses modified for children with autism
| Context/task | Press | Modifications |
|---|---|---|
| Examiner context | ||
| Sorting: Child given directives to complete “jobs” (e.g. sorting utensils) | Compliance-do | Initial directives include demonstration of “job”; additional non-verbal prompts to maximize understanding |
| Bubble blower: Examiner demonstrates “fun” bubble blower then offers child “broken” blower that does not produce bubbles | Frustration | Replaced bubble blower with broken wind-up toy. Fan in bubble toy was appealing; task failed to elicit frustration |
| Marble maze: Examiner & child play with marble maze together | Social play/inflexibility | Substituted Ball Drop set –larger balls (marbles were choking hazard) |
| Goody shelf: Examiner shows child enticing toys; says they can’t play with them yet; gives child paper and crayons | Compliance-don’t | Added toys typically appealing to children with autism |
| Prize/puzzle: Examiner offers choice of prizes; says they can keep prize if they complete (unsolvable) puzzle | Disappointment | Dropped from protocol (children did not understand connection between completing puzzle and earning prize) |
| Restricted interest/preferred object: Examiner allows access to preferred activity/object; after few minutes, ends activity/removes object | Restricted interest | Not in original DB-DOS |
| Parent context | ||
| Don’t touch toy rule: Child told he/she is not allowed to touch appealing toy | Compliance-don’t | Now includes toys children with autism typically find appealing |
| Coloring/crayon clean-up: Parent & child color & clean up together | Compliance-do | None |
| Puzzles: Parent & child complete puzzles with mixed up pieces | Frustration | None |
| Wait: Parent completes forms while child looks at book | Compliance-don’t | None |
| Free play: Parent & child can now play with “Don’t Touch” toys | Social play | None |
| Restricted interest/preferred object: Parent allows access to preferred toy/object or restricted interest; after few minutes, ends activity/removes object | Restricted interest | Not in original DB-DOS |
Open in a separate window
Participants
Participants were recruited through a university developmental diagnostic clinic and had previously been diagnosed with autism and developmental delay. Twelve children ages 36–50 months (M=44.6, SD=5.2, 83% male) assented to the study and their mothers consented to the study. Mothers, between 37–44 years old, represented diverse backgrounds (33% non-dominant race; 10% Hispanic). On average, children presented with moderate-severe autism symptoms as measured by the ADOS-2 comparison severity score15 (M=7.9, SD=1.6, Range=6–10). Non-verbal IQ scores indicated that children presented with mild to moderately delayed cognitive abilities, as previously measured by the Visual Reception subscale of the Mullen Scales of Early Learning (M=61, SD=18.4, range 17–88).16
Measures
We counterbalanced administration of the two standardized assessments: ADOS-2 Module 1 and the DB-DOS (examiner and parent contexts) with modifications identified during pilot testing (Table 1). We selected the ADOS-2 as the comparison context as this measure is already standard practice in autism diagnostics. We scored the ADOS-2 for DB according to the manual (section E) and also applied the DB-DOS coding system to quantify DB exhibited during the ADOS administration. The DB-DOS was scored using standard procedures. During the clinician DB-DOS administration, parents completed the Child Behavior Checklist (CBCL) to provide a comparison to a validated measure of parent reported DB.17 Additionally, the visual reception subscale of the Mullen Scales of Early Learning was administered at study entry to characterize the participant’s nonverbal cognitive ability.16 All direct assessments were administered by a research reliable clinician who was blind to the study purpose, and the assessments were completed with high fidelity (>90% implementation fidelity).9
During the parent context of the DB-DOS administration, parents are given broad directions about the activities (i.e. Please have your child complete these puzzles) and asked to not let their child touch the toys on the shelf until it was time. Each activity is stored in a box with a cue card for the parent indicating the goal of the task. The parent is asked to proceed through the numbered tasks as soon as the timer indicated, even if the task was not complete. Parents are not given specific instructions about supporting (or not supporting) their child’s behavior.
Analysis
We used repeated measures analysis of variance to compare DB-DOS scores across the three administration contexts (DB-DOS examiner, DB-DOS parent, ADOS). We conducted separate analyses for the anger modulation and behavioral regulation domain scores.
RESULTS
Pilot administration
Pilot tests revealed three areas that required adaptation to the DB-DOS protocol (Table 1). First, the children were unable to follow simple directions as the DB-DOS protocol prescribes, and it was challenging to uncouple cognitive ability or receptive language ability from noncompliance. Therefore, we adapted the protocol to include modeling the task and non-verbal prompts to ensure the child understood the expectations. Second, some of the standard test material elicited repetitive or dangerous behaviors (e.g. mouthing/eating small items), and one mother highlighted the inappropriateness of some materials. This resulted in substituting a ball run for the marbles to avoid possible choking hazard, substituting a wind-up toy for the bubble gun to avoid sensory seeking behaviors with the bubble gun, and adding different toys to the “don’t touch” shelf. Finally, one of the pilot children engaged in a specific routine that was described by his mother as a restricted interested, and it was only during this routine (or trying to break this routine) that we observed disruptive behavior. Conversely, one of the pilot children did not engage in many instances of disruptive behavior and his mother reported this was due to the absence of a particular preferred object that her child used to engage in repetitive behaviors. Specifically, the child frequently engaged in disruptive behaviors when denied the preferred object or repetitive behavior is interrupted. Because of these situations, the child’s highly preferred objects related to any restricted interests or repetitive behaviors were included on the toy shelf. This provided an opportunity to restrict access to highly preferred items as well as observe the child’s behavior before, during, and the transition away from having access to these items.
Comparison results from 12 participants
Children presented with a low range of challenging behaviors as reported by their caregivers on the CBCL, total standard score (M=61, SD=6.7, range=51–70). Yet, as shown in Figure 1, children exhibited significantly more problems in behavioral regulation during both the examiner and parent DB-DOS contexts than during the ADOS, F(2,33) = 6.09, p <0.001). A similar trend was observed for problems in anger modulation, but differences were not statistically significant in this small sample, F(2,33) = 2.25, p = 0.13). Scores across DB domains in the adapted protocol were unrelated to cognitive ability (r=−0.3, p>0.05), suggesting that disruptive behavior varies across cognitive ability levels. Two thirds of children with autism (8 of 12) had high scores on at least 1 DB-DOS domain (≥ 1 SD above average scores for neurotypical children for either anger modulation or behavior regulation.7 Yet on the ADOS-2 single item DB index, none had “serious and repetitive tantrums/aggressive behaviors,” fewer than half (5 of 12, 42%) had “occasional tantrums,” and the majority (7 of 12, 58%) exhibited no problem behaviors. Moreover, when DB-DOS codes were applied to children’s behavior during the ADOS, behavioral regulation scores were significantly lower and anger modulation scores tended to be lower than the scores of the same child on the DBDOS, suggesting that the DB-DOS effectively elicits DB in children with autism. Of particular interest, the children who scored as exhibiting no problem behaviors during the ADOS (score of 0 for section E) received moderate-high behavior regulation problems scores across DB-DOS contexts (Parent: M=3.86, SD=3.71; Clinician: M=8.57, SD=5.84; ADOS: 1.86; 2.04). Parents reported that the overall procedures were not only acceptable, but at times validating for clinicians to see the context for and magnitude of DB that their child experiences. Overall the DB-DOS was feasible to administer with the modifications across children, and parents anecdotally reported ease of administration and overall acceptability.
Figure 1.
Disruptive Behavior across contexts
DISCUSSION
The DB-DOS, a developmentally-sensitive systematic observation tool complements a standardized observation measure of autism symptoms (the ADOS) and shows promise for improving the reliability and validity of DB assessment in autistic children. The adapted DB-DOS protocol has demonstrated promise for further refinement towards improving clinical decision-making for children with autism by detecting co-occurring DB syndromes and monitoring DB interventions through a standardized observational paradigm. Using a pragmatic approach, we have shown that a specialized, developmentally-sensitive DB paradigm, such as the DB-DOS, elicits variability and clinically informative patterns of DB in young children with autism. Administering the ADOS and the DB-DOS concurrently, we found that the ADOS was relatively uninformative for DB, based on either its standard DB code or employing DB- DOS codes to observations during the ADOS context, and that the DB-DOS elicited a wide range of DB. These results may provide a richer picture of DB compared to the CBCL, which represents a relatively small range of parent reported disruptive behavior for this sample (SD=6.7).
These findings provide initial evidence that the DB-DOS holds promise as a complementary behavioral measure to the existing Functional Behavior Assessment (FBA). While the FBA process allows for individualized identification of the function of a behavior for treatment purposes, the DB-DOS provides a complementary assessment of the level of DB that is comparable across children and social interactional contexts or as a standardized progress measure for an individual child.7 For example, the DB-DOS many identify pervasive and impairing DB symptoms, but the FBA could help identify the function of these symptoms (either to gain attention or perhaps to avoid a noxious sensory stimulus). Additionally, the DB-DOS provides a standardized context and scoring method that provides increased validity as compared to current parent-report measures, allows for comparison across children as well as within children as they develop over time, and provides a standardized context for systematic observation of parent strategy use.18 The parent context allows for evaluation of variability in expression of DB across contexts with and without parental supports, which will vary based on individual styles and characteristics of the parent.13 Indeed, it provides an excellent “snapshot” of these patterns that broadly mirror home experience, albeit within the artificial lab context. Specifically, for children with autism who present with a high rate of DB, the DB-DOS may be a useful tool to systematically measure changes in DB over time, assess treatment effectiveness, diagnose co-occurring disruptive behaviors and inform treatment planning.
Limitations
The current study provides a proof of concept that the DB-DOS is a useful tool for characterizing and evaluating DB in children with autism. However, this initial step should be considered in light of a few limitations. First, although cognitive ability was unrelated to disruptive behavior outcomes in this sample, it should be noted that children with significant cognitive delays could benefit from additional modifications above and beyond this current adapted DB-DOS.12 Second, although the current modifications do take into account repetitive behaviors and/or restricted interests, it should be noted that this measure is not designed to measure sensory seeking behaviors or sensory aversions. However, these could be added as presses if useful. However, the DB-DOS does provide opportunities to observe disruptive behaviors which may occur when repetitive behaviors or restricted interests are withheld. Third, parent implementation of strategies was not yet rated. As a proof of concept study this context provides an important discussion point, however the validation of coding parent strategies in this unique population requires a separate validation study. Finally, it is important when eliciting DB in young children to have a family centered approach. We provided the parents with a discussion prior to administration around the goals and objectives of the DB-DOS and why eliciting DB in this context was important. Although the parents in this pilot study were all enthusiastic about the opportunity for the clinician to observe some of the child’s greatest challenges, we recognize that not all parents may feel this way. Specifically, future studies should evaluate a wider range of parent characteristics, such as depressive symptoms, anxiety, or socio economic or cultural factors, that may interact with supporting DB, responding to DB, and perception of acceptability.
Future directions
This study provides intriguing preliminary evidence of the promise of a standardized, developmentally-sensitive DB diagnostic paradigm for identifying clinically concerning DB in young children with autism. To optimize clinical informativeness, this standardized multi-method DB measurement system must demonstrate developmental sensitivity, attention to phenotypic expressions of problem behavior in autism, adaptability for children with different cognitive abilities, and provision of specific information about DB functions and contexts. This first requires further refinement to better understand how to adapt this protocol for specific autism symptomology, particularly how to adapt this protocol for eliciting disruptive behaviors specific to interrupting repetitive behaviors and restricted interests or the presence of particularly noxious stimuli. Exposure to sensory triggers or changes in expected routines are factors particularly unique to this population that should be considered when optimizing the DB-DOS for children with autism. This will require a larger scale validation and refinement study that enables the generation of clinical thresholds for autism populations sensitive to specific interests/aversions, age, gender, culture, language and developmental level. Additionally, the potential utility of the DB-DOS to provide a sensitive observational measure of changes in DB may be applicable to other unique populations.
We recognize the time and effort already required to evaluate and diagnose children with autism. In future iterations of this assessment, we propose identification of the items and tasks that are most informative for a wide range of behavior problems or behavior types, perhaps resulting in a short-form screening DB-DOS that will allow for streamlining the assessment process. Along these lines, related recent work is designed to advance integration of pragmatic and developmental considerations.19
Conclusions
Overall the DB-DOS has shown preliminary evidence of an effective tool to elicit clinically informative variability in DB across 12 children with autism above and beyond current available measures. This promising tool may serve as an important component of the existing FBA process as well as inform the diagnostic process and progress monitoring of DB for children with autism.
Footnotes
Authors statement: The authors of this paper have no conflicts of interest to disclose. Each author has reviewed the manuscript and contributed to the content.
REFERENCES
1. Farmer CA, Aman MG. Aggressive behavior in a sample of children with autism spectrum disorders. Res Autism Spectr Disord. 2011;5(1):317–323. doi: 10.1016/j.rasd.2010.04.014 [CrossRef] [Google Scholar]
2. Fodstad JC, Rojahn J, Matson JL. The Emergence of Challenging Behaviors in At- Risk Toddlers with and without Autism Spectrum Disorder: A Cross-Sectional Study. J Dev Phys Disabil. 2012;24(3):217–234. doi: 10.1007/s10882-011-9266-9 [CrossRef] [Google Scholar]
3. Wakschlag LS, Tolan PH, Leventhal BL. Research Review: ‘Ain’t misbehavin’: Towards a developmentally-specified nosology for preschool disruptive behavior. J Child Psychol Psychiatry. 2010;51(1):3–22. doi: 10.1111/j.1469-7610.2009.02184.x [PMC free article] [PubMed] [CrossRef] [Google Scholar]
4. Hanratty J, Livingstone N, Robalino S, et al. Systematic review of the measurement properties of tools used to measure behaviour problems in young children with autism. PLoS ONE. 2015;10(12):e0144649. doi: 10.1371/journal.pone.0144649 [PMC free article] [PubMed] [CrossRef] [Google Scholar]
5. Lord C, Rutter M, DiLavore PC, Risi S, et al. Autism Diagnostic Observation Schedule: ADOS-2. Western Psychological Services Torrance; 2012. [Google Scholar]
6. Wakschlag LS, Briggs-Gowan MJ, Carter AS, et al. A developmental framework for distinguishing disruptive behavior from normative misbehavior in preschool children. J Child Psychol Psychiatry. 2007;48(10):976–987. doi: 10.1111/j.1469-7610.2007.01786.x [PubMed] [CrossRef] [Google Scholar]
7. Wakschlag LS, Briggs-Gowan MJ, Hill C, et al. Observational Assessment of Preschool Disruptive Behavior, Part II: Validity of the Disruptive Behavior Diagnostic Observation Schedule (DB-DOS). J Am Acad Child Adolesc Psychiatry. 2008;47(6):632–641. doi: 10.1097/CHI.0b013e31816c5c10 [PMC free article] [PubMed] [CrossRef] [Google Scholar]
8. De Los Reyes A, Henry DB, Tolan PH, et al. Linking informant discrepancies to observed variations in young children’s disruptive behavior. J Abnorm Child Psychol. 2009;37(5):637–652. [PMC free article] [PubMed] [Google Scholar]
9. Wakschlag LS, Hill C, Carter AS, et al. Observational assessment of preschool disruptive behavior, part I: reliability of the Disruptive Behavior Diagnostic Observation Schedule (DB-DOS). J Am Acad Child Adolesc Psychiatry. 2008;47(6):622–631. [PMC free article] [PubMed] [Google Scholar]
10. Bunte TL, Schoemaker K, Hessen DJ, et al. Clinical usefulness of the Kiddie- Disruptive Behavior Disorder Schedule in the diagnosis of DBD and ADHD in preschool children. J Abnorm Child Psychol. 2013;41(5):681–690. [PubMed] [Google Scholar]
11. Tseng W-L, Guyer AE, Briggs-Gowan MJ, et al. Behavior and emotion modulation deficits in preschoolers at risk for bipolar disorder. Depress Anxiety. 2015;32(5):325–334. [PMC free article] [PubMed] [Google Scholar]
12. Biedzio D, Wakschlag L. Developmental emergence of disruptive behaviors beginning in infancy: delineating normal: abnormal boundaries to enhance early identification. Handb Infant Ment Health. 2018;4. [Google Scholar]
13. Gray SA, Carter AS, Briggs-Gowan MJ, et al. Preschool children’s observed disruptive behavior: Variations across sex, interactional context, and disruptive psychopathology. J Clin Child Adolesc Psychol. 2012;41(4):499–507. [PMC free article] [PubMed] [Google Scholar]
14. Glasgow RE. What does it mean to be pragmatic? Pragmatic methods, measures, and models to facilitate research translation. Health Educ Behav. 2013;40(3):257–265. [PubMed] [Google Scholar]
15. Gotham K, Pickles A, Lord C. Standardizing ADOS scores for a measure of severity in autism spectrum disorders. J Autism Dev Disord. 2009;39(5):693–705. doi: 10.1007/s10803-008-0674-3 [PMC free article] [PubMed] [CrossRef] [Google Scholar]
16. Mullen EMullen Scales of Early Learning. American Guidance Services; 1997. [Google Scholar]
17. Achenbach TM. Manual for the Child Behavior Checklist/4–18 and 1991 profile. Univ Vt Dep Psychiatry. Published online 1991. [Google Scholar]
18. Gray SA, Wakschlag LS. The contribution of developmentally sensitive measurement to assessment of disruptive behavior in young children. Oxf Handb Infant Toddler Presch Ment Health Assess. Published online 2019:371. [Google Scholar]
19. Blackwell C, Wakschlag L, Krogh-Jespersen S, et al. Pragmatic Health Assessment in Early Childhood: The PROMIS of Developmentally Based Measurement for Pediatric Psychology. [PMC free article] [PubMed]
