Nathen's Miraculous Escape

Abstract

A critical review of Edwards’ (1984) “The Effect on Self-Actualization of a Personal Growth Programme Based on Co-Counselling,” considering problem statement and hypothesis, participants and sampling, research design, methods, results, and discussion.  Validity is given extra consideration, including internal and external validity, construct validity, statistical conclusion validity, and cultural validity.  Concludes that the findings are limited in their generalizability because of their preliminary nature and because Edwards falls short on some best practices, but overall the study is credible.

A Critical Review of Edwards “The Effect on Self-Actualization of a Personal Growth Programme Based on Co-Counseling”

Co-counseling is a personal growth technique and movement which holds that emotional trauma is stored in persons in such a way that their behaviors and cognition can be limited or patterned, that emoting in the presence of the loving attention of another person can release the emoter from the hold of that trauma, and that this process allows the emoter to behave and think more freely and accurately, and to be more fully themselves (see e.g. Jackins 1965, 1970, 1973, 1977).  While these ideas are reminiscent of humanistic psychotherapies, such as Carl Rogers’ person-centered therapy (e.g. Rogers, 1980) or Virginia Satir’s version of experiential therapy (e.g. Satir, 1964), there are several important differences, including that co-counseling uses a peer-counseling model almost exclusively, very rarely relying on a professional relationship (Jackins, 1970).

Self-actualization, the holy grail of humanistic psychology, is the enhancement of the individual and their experience of life (Rogers, 1951), or the reaching of one’s potential (Maslow, 1943), and has been found to correlate to self-esteem and rational behavior (Jones, 1986) among other desirable attributes.  If the practice of co-counseling significantly and reliably affects self-actualization, it would be good to know.  To date, the only published quantitative test of the idea that co-counseling increases self-actualization in its practitioners was published by Edwards in 1984.  (Indeed, there appears to have been only been one other quantitative study published that is straightforwardly about the effects of co-counseling, by Stevens, Martina, & Westerhof (2006).)

Edwards used Shostrom’s (1974) Personality Orientation Inventory (POI) to assess the self-actualization of members of a co-counseling class and a matched control group before and after the class, as well as six months later, and found evidence for a greater increase in self-actualization on four of the 12 POI subscales for the co-counseling group compared to the control: Inner Directedness, Time Competence, Self-Actualizing Values, and Self-Regard.   While Edwards’ publication leaves some details to be desired, it appears overall to describe a good attempt at preliminary quantitative research on this topic.

Problem Statement and Hypothesis

Edwards’ article is succinct at 17 paragraphs in three pages, including the abstract.  The abstract is an accurate summary of his method and results sections.  The two paragraphs devoted to the introduction include brief introductions to co-counseling and the POI, and a clear statement of the hypothesis: that participants in the co-counseling group would show increased self-actualization as measured by the POI, compared to the control group.

The brevity of the introduction is due in part to the lack of a body of literature with which to introduce co-counseling, but Edwards has also left out any development of a wider problem that the research was intended to address, or the research question that the hypothesis was intended to address.  TWhile it would have been nice to have this development, the article suffers, but not greatly, from these omissions of development; the value of self-actualization is not obscure, and the likely research question, “Does co-counseling increase self-actualization?” is easily deduced from the hypothesis.

Participants and Sampling

Edwards describes his sampling procedure clearly.  The experimental group was made up of volunteers from his third-year psychology class at a university in South Africa.  The control group was made up of age- and gender-matched students from the same class who participated in other, non-personal growth oriented psychology workshops, rather than take part in the co-counseling training.

Since Edwards doesid not state a research question, his sampling procedures cannot be evaluated in light of a research question.  His use of a convenience sample is acceptable in light of his hypothesis, however, and the state of the research literature on co-counseling; this is introductory research.   ThisIt does, of course, limit the generalizability of his findings, as will be addressed below.

Edwards also doesid not state his population of interest, outside of his class.  We might assume that he is interested in the effects of co-counseling on a larger group, and anticipate the need to attempt to replicate his findingsfor with a more diverse group than 22 students from a South African psychology class in the early 1980s.  It would be good to keep in mind, however, that because of the self-disclosing and intimate process of co-counseling, and because of the time commitment involved (approximately 40 hours in this study), research on co-counseling may always be limited to a somewhat select set of volunteers.

Research Design and Methods

Edwards used a nonequivalent, pre-test-post-test, control-group design with a six-month follow-up to check for stability of differences.   It was quasi-experimental in that participants were not randomly assigned to condition, as discussed above.  Edwards doesid not provide enough detail about the execution of the study to allow for a rigorous replication—he describesd approximately 40 hours of intervention in one paragraph—but to do so would have taken the space of a small book.   He does outline his procedure and give references for the source of his techniques, so that there is enough information for a replication as long as an experienced co-counseling teacher is involved.

This was basically an experimental field study, since the experimental group participated in a co-counseling class voluntarily and did so in much the same way that non-study participants might learn co-counseling.  This design has the potential for both high internal and external validity.  The fact that participants could not be randomly assigned weakens internal validity, but all things considered, this was an appropriate design to test the hypothesis.

Self-actualization, the latent construct of interest, was operationalized as scores on the various subscales of the POI, and Edwards’ procedures for administering the test were described with adequate clarity. Edwards does not, however, present a case for the reliability or validity of the instrument except to say that it has been commonly used in this sort of research, citing Knapp and Shostrom’s (1975) review of such use. This leaves something to be desired, as Shostrom is the author of the POI, and because the POI has been widely used, a more comprehensive case could certainly have been made, including apparent correlations to the POI and its subscales in traits in the population, and some measure of reliability, such as Cronbach’s alpha.

Validity

Internal validity. The primary weakness of this experiment comes from the lack of random assignment to experimental groups.  This introduced two problems for internal validity.  The first and most important is a selection problem, which Edwards addresses admirably in his discussion: There is no way to know whether the greater gains in POI subscale scores in the experimental group were a result of the intervention or from some other factor, or set of factors, shared by that group of people, that had also made them more likely to volunteer for an extensive personal growth process.  We cannot tell, for example, if this possible factor, or set of factors, was responsible for behaviors or learning on the part of the experimental group that would have occurred without the intervention.  This could be a factor in the second threat to internal validity: maturation.  Without randomization we cannot be sure that participants were on the same developmental trajectories, even though they were pulled from the same university course, and  though they were matched for age and gender.  It is possible, for example, that the volunteers for the co-counseling class were for some reason more mature and/or more interested in their own self-actualization, and so their comparatively accelerated growth was a result of that state or motivation.  This consideration is ameliorated to some degree by Edwards’ use of a pretest in which there were no significant differences between the groups on any subscale of the POI.

The fact that both groups were pulled from the same psychology class controls the threat of history to some degree; to a large extent, especially for a field study, the participants were experiencing the same events in their city, university, and class, so those events cannot easily be imagined to have caused the differences between the groups.  On the other hand, this same fact presents the possibility of diffusion, the spread of information between the groups.  Even if the experimental group was asked not to share the information given in the co-counseling class (and it is not clear if they were), they may have done so.  To the extent that the effects measured were due to exposure to co-counseling theory, rather than the experience of co-counseling, diffusion might have muddied results in this event.  However, this probably would have tended to reduce, rather than increase, the significance of effects, causing the experimental and control groups to be more alike, not more different, so this threat to internal validity would have at least not increased the likelihood of type I error.

Edwards’ sample size was adequate for the pre- and post-tests but not large enough to allow for enough statistical power in the case of attrition, which he experienced for his six month follow-up, conducted by mail.  It appears as if Edwards dropped the scores from the participants in the control group that were matched with missing participants from the experimental group, but we are left unsure about what the missing participants’ scores would have been and how they would have affected his conclusions.  This also raises questions for statistical validity, discussed below.

The multiple use of the POI on the same subjects raises the possibility of both test-familiarity and boredom/morale effects, as well as statistical regression.  It may be that a personality or developmental measure such as the POI areis less prone to regression toward the mean than other tests, such as mood or ability measures, but without Edwards’ raw data we cannot know if there was in fact less variability in subsequent tests.  This is not a great concern, however, as the groups were not selected on the basis of their pretest scores.  The familiarity and morale concerns are more concerning, as the POI is fairly long and participants took it three times; they may have gotten bored, paid less attention, or answered as they remembered answering a previous time, desiring to look consistent.

The final consideration for internal validity is the role of the experimenter.  Edwards apparently conducted all of the research on his own, from beginning to end, meaning the experimenter was not blind to hypothesis or condition, and there was nothing resembling a fidelity check on his intervention.  Not only that, but he was teaching a lecture class to the participants in both groups at the same time, allowing the possibility that he may have unintentionally treated the students differently during class, based on what group they were in.   This is not unlikely, considering the amount of extra time he spent with the experimental group, and the intimate nature of that time.  Also, because Edwards was involved in co-counseling before conducting the experiment, he likely had expectations and hopes about the outcome.  All of these factors could have affected his results, and constitute the only major flaw inof this experiment.

External validity. As this research was preliminary, it would be unjust to fault Edwards for failing to test his variables across categories of person, settings, and time periods.  It would also be unwise, however, not to take into account the way these failures affect external validity.  The fact that all participants were undergraduates in the same university, participating in the same psychology class, in the same year, and were likely all young adults and White, greatly reduces the degree to which Edwards’ findings can be generalized to other people in other settings and other times.  There could be a significant variation in effects of co-counseling training on people from other cultures, for example, other age groups or generational cohorts, people who are more educated, less educated, or educated differently, people who live in countries which are not as racially segregated as South Africa was in the early 1980s, or people who experience co-counseling trainings in a setting that is not connected with a university.  Clearly, these findings must be treated as preliminary and suggestive.  To his credit, Edwards does just that, remaining modest in his presentation and not making claims beyond his data.

Construct validity. Edwards gives little attention to construct validity.  It may be, as Edwards claims, that the POI has been widely used for outcome assessments of group trainings (he does offer a citation to that effect), but he does not give a definition of “self-actualization,” a description of what debate, if any, exists over the operationalization of this concept, or a description of the POI itself, such as what populations it has been normed on, or any example items from the instrument.  Additionally, the POI was the only measure used, so construct validity suffers from mono-method bias.  This leaves the possibilities that this way of testing self-actualization the POI found an effect that would not be found by another measure of the same construct, that these college students had a unique response to this particular instrument, or that they learned (perhaps in their psychology class) the value of self-actualization and fell prey to socially-desirable responding.  Also, the four POI subscales which showed greater growth in the experimental group – Inner Directedness, Time Competence, Self-Actualizing Values, and Self-Regard – constitute only a third of the instrumenttest, and may not reflect the entire construct of self-actualization on their own.

Statistical conclusion validity and cultural validity. Low power was a definite threat to statistical conclusion validity.  Power was adequate at pre- and post-test, with 22 participants per group, but at the six-month follow up test each group fell to 11 participants, so that even though the group means appeared to have remained stable since post-test, there was not enough statistical power to say whether the difference remained significant.  Edwards addresses this problem clearly and accurately.

Since some results were positive—the experimental group did show a significantly greater increase in four POI subscale scores than the control group—there is a chance of type I error.  Edwards set his alpha level at .05, so there is a one-in-twenty chance of a false positive for each of the four subscales.  There is also some chance that the negative results—the other eight subscales, and especially the six-month follow-up POI scores, because of the power issue—were false negatives.

While there is no particular reason to expect problems with fidelity or variation in the experimental setting, Edwards also fails to address them as potential problems, so we are left to wonder.  Similarly, though there were no differences between the groups on age, gender, or the dependent variable at pre-test, Edwards does not mention other possible differences between groups on variables like religion or ethnicity.  This means that there is some possibility that some unknown variable could have been distributed differently between the groups, causing some of the differences observed on POI scores.  These elements are threats to both statistical conclusion validity and cultural validity, but again, Edwards points these weaknesses out and does not make undue claims.

Results and Discussion

The results section is adequate but presented in a confused manner.  The data analysis is clear—lots of ANOVAs—but the statistics are shown only in a poorly designed table, not written up clearly in the text of the results section.  Edwards did write a clear general description of his results, but I found it difficult to connect his write-up to the means and F values given in the table.  It took me several tries to understand exactly what he had found, but once I did, it seemed that he had accurately and credibly, if confusedly, presented his findings.

The discussion section is clear and coherent.  Edwards addresses the way POI scores increased significantly for all subjects, regardless of condition, and considers possible reasons for it.  He considers other possible explanations for the greater increases shown by the experimental group.  He considers the problem that not having random assignment to condition posed to the validity of his results, proposing future research to make up for this weakness.  He does not discuss implications for clinical practice or public policy, a good choice considering the preliminary nature of the research. The major glaring omission is that Edwards fails to discuss how his role as experimenter, teacher, and authority figure may have skewed his results.  Overall, however, he presents modest claims for modest but suggestive results, and I found the study to be limited but credible.

References

Edwards, D. J. (1984).  The effect on self-actualization of a personal growth programme based on co-counselling.  South African Journal of Psychology, 14(2), 54-56.

Jackins, H. (1965). The human side of human beings.Seattle: RationalIsland.

Jackins, H. (1970). Fundamentals of co-counseling manual.Seattle: RationalIsland.

Jackins, H. (1973). The human situation.Seattle: RationalIsland.

Jackins, H. (1977). The upward trend.Seattle: RationalIsland.

Jones, A. (1986). Validation of a short index of self-actualization. Personality & Social Psychology Bulletin. 12(1), 63-73. doi: 10.1177/0146167286121007

Knapp, R. R. & Shostrom, E. L. (1975). POI outcomes in studies of growth groups: A selected review. Group and Organisation Studies, 1, 187-202.

Maslow, A. H. (1943). A theory of human motivation. Psychological Review, 50(4), 370-396.

Rogers, C. R. (1951). Client-centered therapy: Its current practice, implications, and theory.Boston: Houghton-Mifflin.

Rogers, C. R. (1980). A way of being.Boston: Houghton Mifflin.

Satir, V. (1964). Conjoint family therapy: A guide to theory and technique. USA: Science and Behavior.

Shostrom, E. L. (1974). Manual for the Personal Orientation Inventory.San Diego: Educational and Industrial Testing Service.

Stevens, N. L., Martina, C. M. S., & Gerben, J. W. (2006).  Meeting the need to belong: Predicting effects of a friendship enrichment program for older women.  The Gerontologist, 46(4), 495-502.