TOC
 

III.B.3. The National Drug-Abuse Policy Delphi: Progress Report and Findings to Date

 

IRENE ANNE JILLSON

 

Introduction

Rationale

Although the abuse of drugs has been recognized in this country for nearly one hundred years, its popularity as a national problem has resurfaced relatively recently. In 1968, former President Nixon declared drug abuse "public enemy number I"; from 1969 until 1974, some $2.4 billion in federal funds were obligated to combat the problem, and an industry was created. This expenditure represents funding of programs that were almost exclusively aimed at heroin abuse, rather than the broad spectrum of drug abuse. Since 1968, drug abuse has been the subject of intense public and private debate. Controversy over the government's appropriate response has ranged from debate regarding drug laws (to what extent different drugs should be controlled, and in what manner), to the basic question of whether or not alcohol programs are to be included with "other" drug programs on the federal level.

During the past ten years, the increased concern and expansion of drug-abuse prevention programs has resulted in a swelling of the ranks of professionals who have developed expertise in this field; however, the use of these experts in policy advice and formulation has been sporadic and unsystematic. At the same time, numerous research and evaluation studies of drug-abuse prevention programs themselves have been carried out. The degree to which resultant data from these studies can be, have been, or should be, used in decision-making at the nation level has never been resolved.

In the fall of 1973, it was clear that the problem of drug abuse had diminished in priority, and that substantial reductions in federal funding were imminent. This may be attributed primarily to the apparent abatement of the heroin epidemic, which had served as the stimulus for increased concern and program funding in the late 1960s. Although the crisis associated with the heroin epidemic may have passed, it is by no means clear that the broader problem of drug abuse has been resolved: polydrug use and alcoholism appear to be increasing; and the abuse of prescription drugs, once the "hidden drug problem," is surfacing in many communities. Such a time of decreased public interest and funding for a yet-existing problem calls for careful consideration of the basic issues, and deliberation of the strategies to be followed, in order to maximize effective use of resources available.

Viewed from this perspective, the procedures utilized by Policy Delphi studies seemed most appropriate to explore national drug-abuse policy options for the next five years. The volatility of many of the issues, most of which involve fundamental value and sometimes moral choices; the diverse backgrounds of those who make or influence policy; the apparent differences in the positions held by various experts and groups; and the apparent inability of past policy studies to aid decision-makers led to the conceptualization and implementation of a national drug-abuse Policy Delphi study. The unusually high response rate, the degree of participation achieved to date, and the interest on the part of federal and state decision-makers has borne out this initial hypothesis.

History

The study described in this chapter was originally conceived in 1973, and designed during the fall of that year, Implementation began in December 1973; the first questionnaire was disseminated in March 1974. The first two questionnaires were developed under a contract funded by the National Institute of Drug Abuse.1 Analysis of the data generated by the second questionnaire, and the further development of the use of the Delphi procedures in the exploration of national drug-abuse policy, as originally conceived, was sponsored by the National Coordinating Council on Drug Education.

The study analyzing the first two rounds was completed and published in December 1974.

Objectives of This Study

There are three primary objectives for this study:

  • To develop a range of possible national drug-abuse policy options
  • To explore applications of the Policy Delphi methodology to this and other areas of social policy
  • To explore the possibilities of applying the technique on an as-needed basis and on an ongoing basis

Since the level of drug abuse in the United States is presumed to be both endemic and epidemic, and since strategies to respond to changes in use patterns need to be both immediate and long range, this study is concerned with ascertaining the feasibility of utilizing the Delphi technique to meet these needs of policy formulation and planning.

  • On an as-needed basis. This would involve the use of a panel of experts who would respond to queries sent as the need arises. For example, if a decision-maker were to be informed that there was a dramatic decline in the number of patients entering treatment programs, a Delphi would be developed to determine the opinion of selected experts with regard to this particular trend.
  • On an ongoing basis. If one agrees that there is an endemic level of drug abuse, then it would seem appropriate to develop an ongoing Delphi study of indefinite duration. This Delphi would be implemented such that questionnaires would be distributed at regular intervals. Since the panel would be of indefinite duration, membership might be fluid. Current trends in the field would be incorporated into the questionnaires, so that there would be a continuous flow of information for the use of the policymaker, and those operating programs of various disciplines in the field.

Study Design

A number of advisers--experts in the field of drug abuse, policy planning and analysis, and the Delphi technique in particular-assisted in developing the study design. To date, they have continued to provide assistance in all aspects of the study. Included are Dr. Norman Dalkey, Engineering Systems Department, University of California at Los Angeles, who originally was instrumental in developing the technique in the late 1950's and who continues to explore its use as part of his decision-theory research; and Dr. Murray Turoff, who has developed the Policy Delphi and is co-editor of this book. Dr. Peter Goldschmidt has assisted in planning and management of this study, and M. Alexander Stiffman has assisted in developing the analytic approach and designing the computer analysis. Dr. Raymond Knowles, Charlton Price, and Anthony Siciargo have assisted in pretesting the questionnaires.

The final study design was based on the premise that policy may be formulated from a number of different perspectives. In designing this Policy Delphi study we are exploring the formulation of national drug-abuse policy from three perspectives:

  • the "top-down" approach - establishing national drug-abuse policy objectives to be achieved over the next five years, based on the respondents' value systems
  • a "bottom-up" approach - identifying factors which control the transition between general population and various degrees of drug use; deciding which of these are important and can be affected by national drug-abuse policy; and determining appropriate policies to affect them
  • an issue-oriented approach - deriving policies from issues which are the subject of current controversy or debate

The underlying thesis in this design is that different decision-makers may formulate policy predominately using one or another perspective, and that this results in distinct types of policy options and considerations which may appear attractive from one perspective, but turn out to be counterproductive from another. For example, setting an objective using the "top-down" approach may result in its achievement becoming a political issue, while an objective that is formulated as the result of a political issue may never be achieved because it is technically impossible even though its achievement is valued.

This top-down, bottom-up approach speaks to the general concept of the Policy Delphi, in that alternative policy options are drawn from a number of different vantage points (it is not simply a statement of objectives, for example) Additionally, we are not interested in consensus per se, but rather, in exploring alternatives, and pro and contra arguments for the alternatives.

For these reasons the outputs from the three approaches used in this study will be brought together so that the panel can identify more appropriate alternatives. We anticipate that this will prove more fruitful in terms of exploring national policy options than either approach alone. A description of the approach, round by round, is given in Table 1.

Respondent Population

A list of more than one hundred highly selected potential respondents was developed from among the most notable "experts" in the field and from those who directly impacted on the field (e.g., police chiefs). Invitations to participate in the study were sent to forty-five persons; the remaining names were held in reserve, as a second series of invitations was anticipated in order to secure twenty-five participants. In fact, thirty-eight individuals (84%) responded positively. Since that time, three respondents have withdrawn from the study owing to a change in career orientation from drug abuse. Needless to say, there was no necessity for a second series of invitations.

As the study progressed past the first two rounds, several additional respondents were added. These additional respondents were selected to represent areas of interest which had developed in the study, but for which respondents had not been initially selected. Experts in alcoholism were added to the panel, for example, because a significant proportion of existing panelists expressed the view that a national drug-abuse policy could not be considered separately from alcoholism. The addition of such experts will allow their views to be added to those of the present panelists, and so provide an appropriate additional perspective. The present panel consists of thirty-nine persons.

Our respondent group represents some of the most respected authorities in the field. They include the Deputy Director of the Alcohol, Drug Abuse and Mental Health Administration, a former director of the Bureau of Narcotics and Dangerous Drugs, officials from the Office of the Secretary, and Office of the Assistant Secretary for Health of H.E.W., notable researchers, treatment administrators, law-enforcement officials, and policymakers in the field of drug abuse. It should be emphasized that participation is voluntary, and that no honorarium is paid to respondents.

The Questionnaires

First Questionnaire

A number of study approaches were explored during the developmental phases preparatory to Round One. The final draft of the resultant questionnaire was pretested, and the entire forecasting section deleted when it was determined that the time to complete the questionnaire was decreased considerably by deleting this section.

Table 1.  THE STUDY DESIGN

Perspective

First
Questionnaire

Second
Questionnaire

Third
Questionnaire

Fourth Questionnaire

Fifth Questionnaire

Final Summary

Objectives

("top down")

List objectives

Rate feasibility, and desirability of objectives

Re-rate feasibility and desirability of selected objectives

Identify any contradiction in objectives and policies formulated by the three approaches

Synthesize a consistent and realistic set of national drug abuse policy options

Write a brief summary of the national drug abuse policy options identified by the study, including the normative forecasts for the key indicators

Identify future policy  research needs

List key indicators

Expand key indicators, perform initial rating

Transition Factors

("bottom up")

List factors, indicate direction

Final rating of factors

Develop policy areas to affect important transition factors which can be influenced by national policy

Policy Issue Statements ("political)

Rate selected policy issues; develop for and against arguments

Final rating of selected policy issues; rate or and against arguments

Add other important policy issues

Rate and give arguments for and against additional policy issue statements

The first-round questionnaire, disseminated in mid-March, consisted of three primary sections:

  1. Development of Objectives. The respondents were asked to develop up to five national policy objectives in the field of drug abuse, given a five-year time frame; and to list up to three key indicators for each objective.
  2. Transition Matrix. Respondents were given a simplified transition model which depicted the flow from one state of drug involvement to another and were asked to list the factors which promoted or inhibited the movement of people from one state to another. (A detailed description of this matrix is provided in a subsequent section.)
  3. Policy Issue Statements. Twelve issues, in the form of should/should not statements, were posed. These had been culled from a potential list of twenty-five issues felt to be current and controversial.
  4. Additional Items. These included a self-rated expertise question; two questions relating to expectations and objectives for participating in the study; a request for responses to a list of definitions; and a request for feedback on questionnaire design and/or content.

Twenty-four of the study's thirty-five respondents (69%) actually completed the Round One questionnaire. The breakdown of returns by category of respondent is shown in Table 2. It should be pointed out that mailing posed serious difficulties both in disseminating and in returning the questionnaires. In several cases it took two full weeks for the questionnaires to arrive by air mail to their destinations; several never arrived, and duplicate packages had to be sent out. For this reason, the deadline for completion of the first-round questionnaires was extended by two weeks.

The absolute range of time for completion of the first questionnaire was fifteen minutes to ten hours. The interquartile range was one to three hours; the median time for completion was 2¼ hours, which was approximately the median we had anticipated. The respondents included in our predesignated policymakers subpanel category spent a median of 3½  hours; the median for all the remaining participants was 2½  hours.

In addition to the substantive-issue questions, respondents were asked to self-rate their expertise in ten drug-related areas (see Table 3). These data were then used to cross-check the categories (subpanels) into which respondents had been placed, and to further analyze responses to particular items. There is one particular point of interest to be noted. As critical as evaluation is held to be in the formulation of national policy, not one of the policymakers rated himself herself as expert in this area.

Respondents were also asked to indicate their expectations for the study, and to list personal objectives with regard to participation. Eighty-two percent of those responding reported that they expected the study to be of direct benefit. There were twenty-eight different narrative responses to this item. Twenty-two saw positive utilization of the results for policy planning, idea exchange, consensus development, etc.; three were uncertain of usefulness, and two were skeptical of its utility. There were fifty-four statements of personal-study objectives; these were distilled to twelve clusters, and are shown as Table 4.

Table 2.
PARTICIPATION AND RETURNS BY CATEGORIES OF RESPONDENTS (SUBPANEL)

Subpanel

Invited

Agreed to Participate

Formally Withdrew

Delphi Panel

Completed Round One

Completed Round Two

Policy Makers

9

9

0

9

5

5

Researchers

12

9

0

9

9

9

Treatment Program Administrators

11

11

2

9

4

6

Criminal Justice Administrators

9

6

1

5

3

3

Other

4

3

0

3

3

2

Total

45

38

3

35

24

25

*As of 1 August, 1974.  Does not include panelists added after that date.

Table 3.
A SUMMARY OF SELF-RATED EXPERTISE BY AREA AND SUBPANEL*

 

Subpanel: Percent who rated themselves "Expert" (1)

Area of Expertise

Policy Makers

Researchers

Criminal Justice System Administrators

Treatment Program Administrators

Other

All Respondents

National Drug Abuse Policy Planning

100

33

33

0

100

50

Prevention

20

22

33

50

0

26

Intervention

0

33

33

75

0

30

Treatment

40

78

0

100

0

57

Law Enforcement

20

11

100

0

33

25

Research

20

100

0

50

0

50

Evaluation

0

78

0

50

33

44

Training

0

33

33

50

0

26

Education

20

56

33

50

0

39

Pharmacology

0

11

0

0

0

4

Number of Respondents

5

9

3

4

3

24

* As of 1 August, 1974.  Does not include panelists after that date.

(1) Percentage of those who responded.

Respondents were asked to comment on a list of standard definitions which had been prepared for use in the study. Twelve respondents commented on these definitions. As anticipated, there was little agreement with these definitions on the part of those who commented. Apart from giving each respondent a common base line, one of the reasons for including the list of definitions was to determine what interest would be aroused. In most of the social-service areas, and particularly in the field of drug abuse, there is much dissension, even among policymakers, regarding critical definitions (e.g., drug abuse, modality types). Clearly it is not sufficient to gloss over this issue continuously in the hopes that at some time in the future standard definitions will somehow be devised and agreed upon by a reasonable majority of those in the drug-abuse field. A Delphi study specifically related to the formulation of standard definitions is presently being planned.

Second Questionnaire

Preparation of the second questionnaire began shortly after the first completed Round One questionnaires had been received. It was decided that, because of the complexity and time required for completion of the transition matrix, this section would be deleted from the second questionnaire and included as part of a subsequent round. The second questionnaire and a summary of the first round results were disseminated in mid-May.

The second questionnaire included two sections:

  1. National Drug-Abuse Policy Objectives. Respondents were asked to rate fifty-five objectives on the basis of feasibility and desirability, and to rate the importance of the key indicators associated with them.
  2. Policy Issue Statements. Respondents were asked to re-rate the original twelve issue statements; rate the narrative comments associated with them; and rate the fifteen new issues suggested by respondents.

There is usually a decrease in response rates for the second round of a Delphi study, particularly those involving voluntary participation. For this reason, and because of the length of the Round Two questionnaire, we anticipated a response rate of approximately 45 percent to 50 percent. In fact, twenty-five respondents, 71%) completed the questionnaire; a most unusual and gratifying response rate, and one higher than that for Round One. (See Table 2.)

The absolute range of time to complete the second questionnaire was 1 2 to 82 hours; the interquartile range was 22 to five hours; the median time to complete was three hours. For the policyrnakers' subpanel, the median was 5(1/4) hours; for all others the median was three hours.

RESPONDENTS' OBJECTIVES FOR PARTICIPATING IN THE STUDY

To test the Delphi technique; determine the value of the process in sharpening views in social policy fields or in bringing forth practical ideas or new insights.

To explore the limits of possible public policy formulation; to learn more about policy formulation.

To be involved in the formulation and development of drug abuse policy; influence policy through identification of critical policy distractions; to share in a process that may lead to wiser process than we now have in the drug abuse area; help develop policy view in a nonpolitical forum.

To see if an consensus is possible in drug abuse policy; see if the group can solve the problem or give direction.

To distill and synthesize the collective thinking of some of the best minds in drug abuse; obtain the benefit of the ideas of the others as a stimulus to my own thinking; to learn what the group knows about, and applies in responding to drug abuse.

To assess the extent to which m views in drug abuse coincide with or differ frosa those of m colleagues; check my own opinions against those of the group.

To clarify m own thinking pertaining to drug abuse policy; get a broader perspective.

To develop priorities for the organization or agency in which I work.

To make a useful contribution to knowledge; help solve a problem; provide ideas that may arise from my special knowledge and experience.

To gain personal enjoyment by responsive discussions.

To satisfy my sense of duty to participate in such studies as an active researcher and/or policymaker.

Third Questionnaire2

For this questionnaire, panelists were asked to respond to two series of questions:

  1. National Policy Objectives. There were twenty-five objectives included in this section; these had to be re-rated because there was a broad distribution of voting responses, differences in voting between policy experts and policy nonexperts, or because original objectives had been combined or divided.
  2. Transition Matrix. In this section, the transition factors suggested by respondents in the first questionnaire were further developed.

The policy issues were not included for consideration in this round in order to maintain expected time for completion to a reasonable level. The data from this third questionnaire, and information gathered during previous studies of this type, will be used as a basis for developing policy and program options in future rounds. The respondents were again sent two copies of the questionnaire, and an introduction and summary volume which included results of the previous questionnaire.

Results

Objectives

First Questionnaire. In the first questionnaire respondents were asked to list up to five national drug-abuse policy objectives (for a five-year time frame), and up to three key indicators for each of these. They listed a total of seventy-eight such objectives; from this list, fifty-five different objectives could be discerned. The 187 key indicators listed by the respondents were culled to 153.

Even after we distilled the original seventy eight objectives to fifty five, there still were similarities between them. Rather than risk a possible misinterpretation of the objectives as stated by the respondents, we decided not to distill the objectives any further. It is difficult to be precise in a Policy Delphi, particularly one in a field as complex as drug abuse; we have therefore emphasized development and creativity.

Second Questionnaire. The respondents were asked to rate the fifty-five objectives derived from their responses to Round One Objectives Section. The two rating scales used were feasibility and desirability. They were also asked to rate the 153 indicators relating to each objective; in this case, the scale used was importance. These scales are shown in Table 5. The objectives were presented in arbitrarily defined categories (prevention, treatment, law enforcement, organization, e.g. simply as a means of ease in completion; these categories had no other significance.

Desirability (Effectiveness or Benefits)

Very Desirable

—Will have a positive effect and little or no negative effect
— extremely beneficial
— justifiable on its own merit

Desirable

—will have a positive effect and little or no negative effect
— beneficial
— justifiable as a by-product or in conjunction with other items

Undesirable

— will have a negative effect
— harmful
— may be justified only as a by-product of a very desirable item, not justified as a by-product of a desirable item

Very Undesirable

— will have a major negative effect
— extremely harmful
— not justifiable

Feasibility (Practicality)

Definitely Feasible

— no hindrance to implementation
— no R&D required
— no political roadblocks
— acceptable to the public

Possibly Feasible

— some indication this is implementable
— some R&D still required
— further consideration or preparation to be given to political or public reaction

Possibly Unfeasible

— some indication this is unworkable
— significant unanswered questions

Definitely Unfeasible

— all indications are negative
— unworkable
— cannot be implemented

Importance (Priority or Relevance)

Very Important

— a most relevant point
— first-order priority
— has direct bearing on major issues
— must be resolved, dealt with, or treated

Important

— is relevant to the issue
— second-order priority
— significant impact but not until other items are treated
— does not have to be fully resolved

Slightly Important

— insignificantly relevant
— third-order priority
— has little importance
— not a determining factor to major issue

Unimportant

— no priority
— no relevance
— no measurable effect
— should be dropped as an item to consider

Confidence (In Validity of Argument or Premise)

Certain

— low risk of being wrong
— decision based upon this will not be wrong because of this "fact"
— most inferences drawn from this will be true

Reliable

— some risk of being wrong
— willing to make a decision based on this but recognizing some chance of error
— some incorrect inferences can be drawn

Risky

— substantial risk of being wrong
— not willing to make a decision based on this alone
— many incorrect inferences can be drawn

Unreliable

— great risk of being wrong
— of no use as a decision basis

The feasibility and desirability ratings of the objectives were analyzed so as to develop a summary list of objectives, and determine to what extent there was consensus or polarization.

Respondents' ratings on the feasibility and desirability of objectives were translated into group scores by summing the scale values and dividing the total by the number of ratings. This procedure, it should be noted, treats nominal scales as interval data. The feasibility and desirability scores were used to categorize objectives as follows:

Group Score

Feasibility

Desirability

Less than 1.80

Highly feasible

Highly desirable

Equal to or greater than 1.80 but less than 2.6

Feasible

Desirable

Equal to or greater than 2.60 but less than 3.40

May or may not be feasible

Neither desirable nor undesirable

Greater than 3.40 but less than or equal to 4.20

Probably infeasible

Undesirable

Greater than 4.20

Definitely infeasible

Highly undesirable

Objectives were first grouped on the basis of their feasibility and then sorted on the basis of their desirability. This produced the rating of objectives depicted in Table 6.

The twenty-five objectives which scored Highly Feasible and Highly Desirable, or Feasible and Highly Desirable, are shown as Tables 7 and 8, respectively.

No objectives were rated "Definitely Infeasible" and none was rated as either "Undesirable" or "Highly Undesirable." These results indicate that the majority (55%) of the objectives listed were rated at least "Feasible" and "Desirable." The following items or sets of items deserve special attention because of the distinctions in rating patterns. The feasibility of twenty-one objectives was indeterminable, either because there was polarization (with some respondents rating an objective feasible while others rated it infeasible); a broad distribution (with respondents voting approximately equally for four or more of the five scale values); or truly indeterminable (with the modal response being "May or may not be feasible"). In only eight of the twenty-one objectives which scored "May or may not be feasible" was this the modal response; in the case of eleven objectives the reason was that the voting was either polarized or broadly distributed, as 'fable 9 shows. All seven of the objectives which respondents scored "Neither desirable nor undesirable" were either polarized or of a broad distribution.

By reviewing the frequency distribution and the scale scores we were able to identify objectives in which there was a significant voting difference between those who rated themselves experts in national drug-abuse policy and those who did not. Table 10 lists these items. Some of the differences were in items that could be of major importance in the formulation of a national drug-abuse policy; most of the differences had to do with the feasibility of attaining objectives.

Table 6.
NATIONAL DRUG ABUSE POLICY OBJECTIVES: Summary of Feasibility and Desirability Ratings

FEASIBILITY

DESIRABILITY

Highly Desirable

Desirable

Indeterminate Desirability

Undesirable

Highly Undesirable

Highly Feasible

5

0

0

0

0

Feasible

20

5

1

0

0

Indeterminate Feasibility

9

6

6

0

0

Infeasible

2

1

0

0

0

Highly Infeasible

0

0

0

0

0

Table 7.
OBJECTIVES VOTED "HIGHLY FEASIBLE" AND "HIGHLY DESIRABLE"
(in decreasing order of desirability)

Objective

Feasibility Score

Desirability Score

To conduct research on treatment modalities and effectiveness.

1.50

1.14

To train in-line treatment personnel to enhance their skill in helping the drug dependent person

1.77

1.27

To have available more adequate epidemiological estimates of a) prevalence, by drug type; b) incidence, by drug type and individual characteristics; c) discontinuance, by drug type

1.73

1.41

To develop, validate and disseminate information on efficacious programs in vocational rehabilitation, and early intervention.

1.79

1.46

To increase research into brain chemistry and psycho-social correlates and clinical-social treatment.

1.70

1.70

In five cases, the modal response of the policy experts was "May or may not be feasible," while nonexperts voted the same objective feasible. These related to major strategies such as "to develop adequate alternative models in prevention and phased intervention," to reduce prescribed use of psychoactive drugs diminish misuse by physicians," and even "to incorporate drug treatment into standard health delivery systems." Since these objectives were all held to be at least desirable, and since one is unlikely to propose objectives one is unsure are achievable, bringing to light this additional information may broaden the policy options available to decision-makers. Alternately, it could be that the view of nonexperts in these cases is overly optimistic. In one case ("to consider social-action approaches as alternatives to treatment...") the modal response of the nonexpert was "May or may not be feasible"; the policy experts were sure it was "Probably feasible."

Policy experts and nonexperts differed on two objectives which represent a major effort in the present national drug-abuse prevention strategy. Policy experts scored "to reduce the supply of drugs available for abuse" as "Feasible," while nonexperts were less certain, scoring it "May or may not be feasible." The reverse was true of the objective "to establish an effective social-rehabilitation system for drug abusers who have become desocialized." Policy experts were not sure if this objective was attainable and scored it "May or may not be feasible"; nonexperts, on the other hand, scored it "Feasible."

Only one objective exhibited a voting difference between policy experts and nonexperts on both feasibility and desirability. This objective ("to develop a group with a primary interest in the problems people have with drugs") was scored "Undesirable" and "May or may not be feasible" by policy experts, but "Desirable" and "Feasible" bv nonexperts.

Although there was a big difference in the desirability score between policy experts and nonexperts on the objective "to train in-line treatment personnel to enhance their skill in helping the drug-dependent person," this was mostly in agreement with 40 percent of experts and 100 percent of the nonexperts voting this objective as "Highly desirable."

Objectives that score "Probably infeasible" or "May or may not be feasible" (and this scale value was the modal response), were dropped from consideration, unless there was a significant difference in voting between policy expert and nonexpert.

Third Questionaire. In this questionaire, twenty-five objectives were listed, which required revoting (desirability and feasibility). Objectives are presented for revoting because of polarization on the part of the panel; because there was a broad distribution of voting responses; or because there were differences in voting between policy experts and policy nonexperts. In some cases, original objectives were combined or divided after respondents' comments had been reviewed; in this instance, voting was required on the newly developed objective.  The remaining objectives will be held over until a later round. Consideration of the key indicators associated with objectives rated at least feasible and desirable will also be held over to a subsequent round.

Transition Model and Matrix

First Questionnaire. Social policy is the result of multiple interacting forces. Policy is often seen as being based on advocacy rather than derived from a careful analysis of empiric findings. Although policy may have to be developed even in the absence of information, a rational examination of the bases on which policies have been built is a fruitful way of providing the policymaker with insight to develop more appropriate policies.

In the case of drug abuse, the factors which cause people to pass through various states of drug dependence can be systematically examined. Such examination allows the policymaker to estimate the importance of specific variables, and the extent to which they are subject to his influence. A systematic examination of factors also allows any counterintuitive effect of the variables to be brought to light.

In this part of the study we hoped to elicit from respondents factors which control the rates of flow from general population through the various states of drug abuse. The simplified model shown in Fig. 1 developed for this purpose was intentionally simplified to allow for examination of the five transition states included in the matrix. More complex (and probably more realistic) models would have diverted attention from the question at hand. The five transitions in which we are particularly interested are:

Respondents were asked to list the factors which affected each of the transitions and state whether a specific factor increased (promoted) or decreased (inhibited) the rate of flow of individuals from one state to another.

The number of factors listed by respondents ranged from a low of two to a high of over forty; we identified a total of 128 distinct factors.

Using the criterion that a factor must register three votes for a single transition, twenty-five significant factors were identified from our total list of 128 factors. The twenty-five are shown in Table 11, the vote is shown by transition state. The table shows the number of respondents who thought that the factor increases the transition rate from one state to another, the number who said it decreases the rate, and a residue who did not indicate direction. Because the number of votes is small, and also because the interpretation of respondents' indication of direction was sometimes difficult, reference will be made only to the total number of votes indicating that a particular factor affected a given transition. It should be noted in passing, however, that for some factors respondents appeared to disagree on the direction in which a factor affected a transition rate. A wide range of factors made the listing; some of them were interwoven into the fabric of society, some were clearly interdependent, and in other cases any relationship between factors was less clear-cut. The dominant factor was "the availability of drugs," which scored half again as many votes as the second factor, "peer pressure," which in turn scored almost half as many votes again as the third factor, "enforcement activities/law-enforcement pressure."

The top ten factors, arranged in rank order of voting across all transitions, together with the number of votes cast for the factor not affecting a particular transition are shown in Table 12.

It is important to note that the votes of "no effect" were obtained from respondents who completed the matrix and who specifically recorded the fact that a particular transition was not affected by the factor they had listed. This means other respondents may also not think a particular factor affects a transition, and so in order to balance the picture the number of nonvotes is also shown. Nevertheless, it is interesting to note that even for the most influential factor ("availability of drugs") some respondents specifically said it did not affect transition #1 (general population to potential user) and transitions #4 and #5 (occasional user to drug-dependent person, and drug-dependent person to formerly dependent person). The factor with most disagreement was "drug-abuse education" (all transitions). We presume that this item refers to drug education as it is practiced now, rather than what its potential impact might be if practiced "correctly." It is particularly noteworthy that no less than eight respondents specifically said that "availability of community-based treatment" did not affect the transition from general population to potential user, and from potential user to experimental user. This number of "no effect on " votes was twice as high as for any other factor, and particularly interesting in that not a single respondent stated that the factor actually affected these two transitions.

The wide range of impacting factors and the fact that some of them are closely related to the kind of society in which we live illustrates the problem of creating specific drug-abuse programs.

Second Questionnaire. The transition matrix was not considered in this round.

Third Questionnaire. In this questionnaire we examined the variables that influence the transition from one state of drug involvement to another. Respondents were asked to rate the twenty-five variables that were most frequently cited in response to the first questionnaire; ratings included estimating the importance of each factor in controlling a particular rate of transition, and the extent to which these factors can be influenced by the national drug-abuse policyrnaker (manageability). These data can then be used by respondents to reformulate priorities for national policy, and develop programs from the objectives and policy-issue-statements sections of this study. Respondents were also asked to rate the criticality of the five separate transitions from one state of drug involvement to another, in terms of the priority each transition should receive as part of the national drug-abuse policy. Respondents did this by allocating one hundred points over the five transitions so as to reflect their priorities. One of the intents of this question is to develop alternative methods of determining program-funding priorities.

Table 12
TOP TEN FACTORS ACROSS ALL TRANSITIONS

Factor

Response

Transition

#1 (General Population to Potential User)

#2 (Potential User to Experimental User)

#3 (Experimental User to Occa-sional User)

#4 (Occasional User to Drug Dependent Person)

#5 (Drug Dependent Person to Former De-pendent Person)

Availability of illicit/euphoria producing/anxiety reducing drugs

Affect
No Affect
No Response

10
3
4

13
0
4

13
0
4

10
2
5

8
4
5

Peer pressure

Affect
No Affect
No Response

6
1
10

8
0
9

8
1
8

7
2
8

6
3
8

Enforcement activities/ law enforcement pressure/application of legal sanctions

Affect
No Affect
No Response

4
2
11

5
1
11

5
1
11

5
1
11

6
0
11

Drug abuse education programs

Affect
No Affect
No Response

6
2
9

7
1
9

5
3
9

2
6
9

4
4
9

Proportion of drug users in peer groups

Affect
No Affect
No Response

5
0
12

4
0
13

4
1
12

4
1
12

4
1
12

Availability of effective stress-relieving alternatives to drug use/lack of boredom

Affect
No Affect
No Response

5
1
11

5
1
11

5
1
11

3
2
12

3
3
11

Supply of drugs/heroin

Affect
No Affect
No Response

2
1
14

4
0
13

3
0
14

2
1
14

9
0
8

Age

Affect
No Affect
No Response

4
0
13

5
1
11

4
1
12

3
2
12

2
3
12

Severity of legal sanctions/laws against use/legal prohibition of drug use

Affect
No Affect
No Response

4
1
12

5
0
12

3
2
12

1
4
12

4
1
12

Availability of community based treatment/access to treatment centers

Affect
No Affect
No Response

0
8
9

0
8
9

2
5
10

5
3
9

9
0
8

Policy Issue Statements

First Questionnaire.  For the first questionnaire, panelists were asked to respond to twelve should/should not statements, constructed on current policy issues in the drug-abuse field, and to indicate the importance of these statements. Respondents were also asked to list issues they thought were sufficiently important to be included in the study.

Pro and contra arguments for these issue statements were to have been elicited as part of the second questionnaire; in fact, the respondents used their first questionnaire comments regarding each of the statements to do just that. A total of 184 separate comments were made, and were fed back as part of the second questionnaire; respondents were asked to rate their importance.

A total of forty-six additional policy issues was suggested by respondents in the first questionnaire; once again, these seemed to exemplify the diverse and complex nature of this field. From this list we were able to formulate an additional sixteen policy issues in the form of should/should not statements.

Second Questionnaire. The issue statements section of the second questionnaire included a statistical summary of responses from the first questionnaire, and a list of the narrative comments made by respondents from each issue statement. The respondents were asked to revote the issue statements, and rate the narrative comments with regard to importance. In addition, thirteen of the issues posed by respondents themselves in the first questionnaire were fed back in the form of should/should not statements for initial voting. It should be noted that three of the original issue statements were rewritten for clarity, following the feedback from the respondent panel.

In preparation for this questionnaire, the analysis or the policy issue statements was confined to comparing the group's response in Round One and Round Two on those items which were common to both rounds (no major shifts were observed in the group's position); and to comparing the Round Two responses of self-rated policy experts to those of nonexperts. Both rating responses and importance ratings were compared.

Respondents were almost unanimous in thinking that "Treatment programs should offer treatment for more than one type of dependence within the same facility" and that "The federal government should allocate funds to community health centers so that these centers can offer treatment for drug abuse." In this round, substantially more respondents felt that the private insurance plans should be encouraged by the federal government to provide coverage for the treatment of drug dependence than in Round One. In Round One, 42 percent of respondents felt that marijuana should be legalized. This support dropped to 32 percent in Round Two, but 81 percent of respondents felt that "The personal use of marijuana should be decriminalized" (a new item).

Issues were classified according to their importance, determined by means of the group importance score. Table 13 lists all thirty-one issues in order of importance; the most important being "There should/should not be a national registry of drug-dependent persons... ." Also shown in `fable 13 is a comparison of the way self-rated policy experts and nonexperts voted on the Round Two issue statements. Really major differences were observed on only three items. The greatest difference was in relation to the issue "Patients who enter treatment should/should not be required to remain in treatment for a minimum amount of time"; 70 percent of policy experts voted should while 77 percent of nonexperts voted should not. Only 56 percent of policy experts voted should compared to 96 percent of nonexperts on the issue "When an individual is referred for treatment in lieu of incarceration, he should/should not have the right to choose the treatment he prefers." On the other hand, 100 percent of policy experts voted that "Regulations should be passed by Congress prohibiting any and all alcohol and tobacco advertising via any media," compared to only 58 percent of nonexperts.

There were four issues that exhibited marked differences between policy experts and nonexperts in the importance of issues (see Table 13). Policy experts scored the importance of the issue "Treatment programs should/should not offer treatment for more than one type of dependence within the same facility" 2.60, while nonexperts thought it considerably more important, scoring it 1.67. A similar situation pertained to the issue "The manufacture and sale of cigarettes should/should not be outlawed." Although both policy experts and nonexperts considered the issue not to be particularly important, experts scored 3.40, while nonexperts scored 2.42.

Marked differences were also observed between self-rated policy experts and nonexperts in the importance scores of these issues:

Preliminary Epilogue

Resource Requirements

One of the advantages of the Delphi technique as a tool in policy analysis is its minimal cost for maximum output. The costs for completion of a Delphi study such as this one can range from $15,000 to $40,000 for a nine- to twelve-month effort, depending upon staff and direct-cost expenditures required. For example, if the effort is included as part of ongoing staff assignments, then staff and space costs may not be directly chargeable; if computer services are available, then a sizable cost category is deleted. The amount of data which may be derived, and the opportunity afforded to facilitate a "discussion" of the issues by divergent experts in the field, render the technique unusually cost-effective.

Considerable time should be spent in coceptualization of the study design and development and pretesting of the questionnaire. In any area as complex and diffuse as drug abuse, the study-design team needs to allocate substantial effort to this phase of study development.

Applications

The relative success of this National Drug-Abuse Policy Delphi has resulted in considerable interest in utilization of the technique not only in the drug-abuse field, but in other social policy areas as well. The opportunities it affords for idea exchange among diverse professionals and interest groups; and the continuous flow of significant data for policy review are but two of the positive attributes of the method. The potential for its application is extensive; as this is the first study of its type, all of us who are interested in its future application can profit from the lessons learned from this effort.

The process of conceptualizing and analyzing policy options is supremely complex; it may be that the Delphi policy method will be a significant advance in the field of applied decision theory and policy analysis, as it relates to the social policy area in particular.

Prospectus

This phase of the study was completed in December 1974. During the succeeding rounds, the objectives will be further summarized; the policy issues on which there is significant divergent opinion will be explored; and policy options will be developed from the objectives, policy issues, and transition factors. An interactive conference will be held at the conclusion of the study; part of this conference will involve introduction of the computer conferencing technique to respondents.

We shall evaluate the present effort to determine to what extent the study objectives were met, and to what extent the respondents' objectives for participating in the study were met. The preliminary steps in this evaluation have already been taken: in the first questionnaire, respondents were asked to list their objectives for participating in the study, for example. At the conclusion of the study, the respondents will be asked to measure the degree to which their objectives have been reached and whether they might have developed other objectives during the course of the study. In addition, they will be asked to evaluate the study on the basis of questionnaire design, content, and other relevant areas. The results of this evaluation will be utilized in developing an ongoing interactive policy planning system which the author is presently designing, as well as other specific studies which are expected to stem from the present effort.

The evaluation of the impact of a study such as this is a much more complex problem, but one which we believe is ultimately of more significance. We have just begun to develop plans for a long-term evaluation of the study. This will include, for example, as assessment of the degree to which study results were reviewed and considered in the formulation of national drug-abuse policy.

Acknowledgments

I should like to acknowledge the extraordinary gratis assistance of Dr. Norman Dalkey and Dr. Murray Turoff; and that of Dr, Peter Goldschmidt and M. Alexander Stiffman of the Johns Hopkins University; without the support and dedicated efforts of these individuals, and the 39 respondents, the study would not be possible.

I should also like to thank the National Coordinating Council on Drug Education for sponsoring the continued development of the study design as originally conceived and implemented.

1 NIDA Contract Number B2C-5352/HOIMA-2-5352.

2 First Round-National Drug-Abuse Policy Delphi.

 

 

 

 

 

 

TOC

©2002 Murray Turoff and Harold A. Linstone