Current Issues in Education :: Volume 1, 1998 :: Number 6

Search:

This article has been retrieved times since November 18, 1998

Our paper has been formatted with extensive use of hypertext and frames.  Our goal is to show that the Internet affords the presentation of a substantially larger amount of information concerning research results than has previously been possible. Hypertext can provide access to multiple sources of information at once, as well as at will. The use of frames is central to our position as frames allow the viewer to examine multiple sources of information at once without disturbing the continuity of the reading process. Contrast this presentation with the standard paper journal, where the reader must stop reading, turn to the page containing the graph or table, and then return to the text. Examining information contained in two tables on different pages is even more cumbersome.  Similarly, in a non-framed format the reader is forced to click a link, have it appear on top of the text, and click another button to get back to the original text.

Traditional quantitative analysis emphasizes highly focused examination of data that may rely on unexamined assumptions. This approach is reinforced by the use of paper based publishing in which terseness and extreme reduction may be valued over rich description. While focus and terseness are appropriate at some stages of research, educational research often seeks the answers to broad questions that should be considered and reported in great depth. This document aims to illustrate the richness of quantitative analysis that can be obtained through the principals and techniques of exploratory data analysis as well as the richness and detail of reporting obtainable through the use of hypermedia documents. This is accomplished in the context of reanalyzing data originally discussed by Gelb and Mizokawa (1986). These authors examined the relationship between levels of social/demographic variables and rates of classification in special education program. The reanalysis presented here concludes that the original results of Gelb and Mizokawa (1986) were understated because of their failure to examine and report the data in these more detailed ways.

    In 1986, Gelb and Mizokawa published an article in the American Educational Research Journal entitled "Special Education and Social Structure: The Commonality of Exceptionality." These authors examined the degree to which economic and social variables were related to the frequency of categorization of special education students at the state level. These authors were particularly concerned with the possibility of varying relationships between so-called "objective" or physical impairments such as hearing impairment, and "subjective" or psycho-educational categorizations such as labels of mental retardation or giftedness. On the basis of a series of multiple-regression analyses, Gelb and Mizokawa (1986) concluded that levels of prevalence of economic and social variables are related to classification rates for the subjective categories of educable mental retardation (EMR) and learning disabled (LD), and that these patterns are qualitatively different from the pattern of variables related to classification rates in objective classification categories. The authors concluded, "It appears then, that these two subjective classifications of exceptionality identify individuals whose deficits are qualitatively different from those labeled under the objective, more normative classifications" (p. 552). From this they inferred that "Whatever the causes of mild learning difficulties, their association with social variables indicates that such problems are not qualitatively similar to the objective disabilities that Public Law 94-142 lumps them with." (p. 554).

    While the tone of these conclusions is consistent with the views of a number of scholars (e.g., Smith, 1982), others such as Behrens & Smith (1996) have raised questions concerning the validity of the original analysis and demonstrated some possible avenues of reanalysis using techniques of exploratory data analysis (Behrens, 1997a; Mosteller & Tukey, 1977; and Tukey, 1977). Nevertheless, a complete reexamination of these data and conclusions has not been undertaken.  The goals of this paper are threefold. First, we seek to reassess the validity of the claims put forth by Gelb and Mizokawa by reanalyzing the data they originally published. Second, we seek to demonstrate the value of graphical and exploratory data analysis and discuss its usefulness in educational research. Because exploratory data analysis relies on the intensive use of graphical information and numerous analyses, it has seldom appeared in the published educational literature. Behrens, Dugan, & Franz (1997) argued that the format constraints of paper media have contributed greatly to a research reporting genre that is excessively terse and insufficiently rich compared to other, often qualitative, approaches. Our third goal then, is to demonstrate the value of hypertext media to report rich descriptions of educational data that are generally unavailable to authors working in paper-oriented formats.

 

    This document is organized into three parts. First, we review the original analysis presented by Gelb and Mizokawa (1986); second, we provide our own reanalysis of the data based on the principles and techniques of exploratory data analysis. In each of these sections we rely heavily on the capabilities of hypertext to present numerous graphical and algebraic summaries. A third section summarizes the findings and addresses the value of the electronic format provided here.

The data used by Gelb and Mizokawa (1986) consist of data from each state as well as the District of Columbia. The authors, however, removed the District of Columbia from the sample due to what they considered "a significant masking effect on the relationships between social variables and the incidence of educable mental retardation (p. 549)." The exact diagnostics used to support this decision were not reported. The goal of the Gelb and Mizokawa study was to determine whether the subjective classifications were socially constructed. They sought to accomplish this by examining correlations between prevalence estimates of categorization and measures of social prosperity. They wrote:

1. Computing the correlation between each special education categorization variable and each independent variable.
2. Identifying the special education categories with more than one significant correlation with the independent variables.
3. Conducting multiple regressions to predict prevalence rates for disability categories identified in step 2.

Having identified the categories of EMR and LD as related to some subsets of the demographic variables, Gelb and Mizokawa (1986) conducted multiple regression analysis to determine which of the demographic variables were associated with levels of special education categorization, while controlling for the inter-correlations amongst the demographic variables. To reduce the number of variables which would enter the computation, the authors used each of the race-rate variables in their original form, along with a series of newly constructed variables that were combinations of existing SES and social deviation variables. The new variables were computed by adding standardized versions of the original component variables as follows:
 

New Social Variable	Components (standardized before addition)
Positive Socio-Economic Status (SES +)	personal income + educational cost
Negative Socio-Economic Status (SES -)	infant mortality rate + children poverty rate + public aid level
Social Deviance (SD)	corporal punishment rate + incarceration rate

To explain the rate of EMR these authors constructed a model which included the new composite variables (SES +,SES -, and SD) along with the racial proportion variables of black and Hispanic residents, because they were significantly related to EMR rates in the original correlation table. Using a backward elimination strategy with a cut-off of p= 0.05, SES  +, SES -, social deviance, and the racial proportion variable of  Hispanic residents were retained but according to the researchers the variable indicating proportion of Blacks was non-significant and removed.  The remaining four variables accounted for 72% of the variance in EMR (F = 28.94, 4, 45 df, p < 0.001).

Gelb and Mizokawa (1986) conducted a similar regression analysis predicting LD prevalence rates using the same five variables (SES+, SES-, social deviance, proportion Black, and proportion Hispanic). However, when the backward elimination strategy was used, only the three composite score variables were retained (SES+, SES-, and social deviance), with these three variables accounting for 32% of the variance (F = 7.44, 3, 46, df, p < 0.01). In light of these analyses, the authors concluded that "Policy should be changed to direct attention away from presumed deficits of individuals and towards identification and treatment of the full range of factors that may cause mild problems (p. 543)."

Rationale for reanalysis
The methods used by Gelb and Mizokawa (1986) are prototypical for correlational analysis in quantitative educational research. They began with hypotheses that were well formulated but vague in the specification of the size and pattern of correlations (e.g., "some socio-economic variables will be correlated"). Therefore, while they had a guiding hypothesis or theme, they were essentially on an exploratory search for patterns of possible relationships. Under such conditions, they relied heavily on the simple Pearson product-moment correlation which is well known to be easily misrepresented in cases with outliers, floor or ceiling effects, curvilinearity, or heterogeneous variance. It is important that each of these possibilities be excluded, and that appropriate rescaling occur when such difficulties arise. However, the authors make no reference to these possibilities, and would likely have little room to report such results in the print format they used.

In addition to these computational possibilities, there are theoretical concerns with these results as well.  We are struck, for instance, with the fact that categorization rates for EMR and LD are correlated with socio-economic variables while those for SED and Gifted are not. This is curious since EMR is perhaps the most "objective" of the subjective categories insofar as there are relatively clear guidelines for levels of performance on well-known tests and they represent a degree of severity beyond that of LD or SED differences. If the hypothesis of differential processes for subjective and objective classification rates holds, it should hold uniformly across the subjective categories.
 

The approach to the reanalysis taken here follows the approach of exploratory data analysis (EDA) articulated by John Tukey and his colleagues (see especially  Mosteller and Tukey, 1977, Tukey (1977)) which has been more recently discussed by Behrens (1997) and Behrens and Smith (1996). Behrens described EDA as:

characterized by (a) an emphasis on the substantive understanding of the data that address the broad question of "what is going on here?"; (b) an emphasis on graphic representations of data; (c) a focus on tentative model building and hypothesis generation in an iterative process of model specification, residual analysis, and model respecification; (d) use of robust measures, reexpression, and subset analysis; and (e) positions of skepticism, flexibility, and ecumenism regarding which methods to apply (pp. 131-132).

Following the guidelines recommended by Tukey (1977) and others (e.g., Mosteller and Tukey, 1977) we begin by examining the univariate distributions of each of the variables of interest. For each variable, histograms, box-plots, and normal-probability plots were observed. This is important because such an analysis helps us build a detailed mental model of the data, and may raise a flag concerning possible anomalies in the data.   Severe skew can cause the mean to be severe, and thus be a misleading estimator of the center of the distribution.  In addition, skew in a univariate distribution is often associated with curvilinearity in bi-variate (scatterplot) distributions.  Such curvilinearity leads the common correlation to underestimate the true strength of the relationship between the two variables.

Considering the subjective categorization variables, the LD distribution was relatively bell-shaped, while the distribution of EMR showed moderate positive skew. The distributions for both SED and Gifted both were skewed severely positive. Next, we examined the distributions of the 13 social demographic variables.

The distributions of data for the ethnic percentage variables were all markedly non-normal. Percent Whites showed negative skew while the percent Black variable was positively skewed. Distributions for American Indian, and Hispanic were very positively skewed. The percent Asian American distribution revealed that all states had below ten percent prevalence except for Hawaii the population of which consisted of almost 70% Asian Americans.

Turning to the social deviance variables, the distribution for suspensions was relatively normal, while the distribution of inmates was moderately skewed and the distribution of corporal punishment was very severely skewed. The distributions for the socioeconomic variables were relatively bell shaped for infant mortality, income and public aid. The poverty variable was relatively normal exhibiting a slight skew to the right. In contrast to these variables, educational cost per pupil was positively skewed with an extremely high value representing the state of Alaska.

Reexpressions and consideration of outliers
The next step in this reanalysis was to determine whether rescaling the more skewed distributions would provide us with information on the underlying patterns that existed in the data that otherwise would be hidden because of their non-normal behavior. Reexpressions (also called transformations) are undertaken in EDA as a method of rescaling toward symmetry.  This is very often needed when little attention is given to the form of the distributions in advance of the analyses.  We first looked at the transformed distributions of the ethnicity variables, then proceeded to examine the social demographic variables that would benefit most from a transformation.  We then concluded with an examination of the four subjective classification variables.  We followed the procedure of raising the data to various powers along the "ladder of powers." Behrens (1997a) and Behrens and Smith (1996) discuss this approach and its application in educational and psychological data.

Ethnicity
The ethnic variables were predominantly skewed to the right, except for percent of Whites which was negatively skewed. The percent of Asian Americans variable posed a problem in that it did not suffer from extreme skew, but rather the presence of an extreme outlier in the case of Hawaii.  Because the extremeness of the value and the fact that the social and economic dynamics in Hawaii are different from those on the continental United States, it was decided to restrict the analysis to the 48 contiguous states.  This setting aside of extreme values allowed for the potential development of an appropriate quantitative model for the 48 states as opposed to developing a clearly inappropriate model for all 50 states.  Once this was accomplished a natural log transformation was carried out producing the LNAsin variable. The percent Hispanic variable benefited from a natural log transformation.   The percent of Whites variable did not benefit from any attempts of transformation.  The American Indian  variable was initially transformed using a log base10 and then a square root; however, neither of these transformations moved the distribution to a more symmetric distribution, forcing the use of the untransformed variable.  Transformation of the percent Black variable did not appear to improve the shape of the distribution, so its original scale was retained.

Social Deviance and Economic Variables
Of the three social demographic variables associated with social deviance only the corporal punishment variable was severely skewed and in need of transformation.  In this case, a log transformation caused the distribution to become negatively skewed, so a square root of corporal punishment was used.  This did  not completely remove the positive skew; however, it did shift the distribution to the right allowing for increased potential of interpretive ease. After examining the five socioeconomic variables it was determined that no variables were skewed severely  and thus, they were not in need of transformation.

Subjective Categories
From the examination of the variables, as seen from the distributions presented above, three of the four subjective classifications category (EMR, SED, and Gifted) were positively skewed with LD retaining a normal bell shaped distribution.  The three non-normal distributions benefited from natural log transformations; LEMR, LSED, LGFTD.

Summary of initial analyses
After examining the data used by Gelb and Mizokawa (1986) it was determined that several of the variables were in need of rescaling. These transformations were undertaken in hopes of improving the description of the data.  Examination of the histograms revealed two extreme outliers: the states of Hawaii and Alaska.  While Gelb and Mizokawa (1986) removed the District of Columbia from the analysis because it represented an extreme outlier that would mask other important trends if included in the analysis, they failed to consider (or report) the difficulties associated with Alaska and Hawaii.   While their presence may be valuable for some political purpose, for the purpose of building descriptions of the relationships among the variables, Alaska and Hawaii present obviously different processes than those that occur in the 48 continental states.   Accordingly we restricted the rest of our analysis to the continental states where the social and economic processes are more homogeneous.  Two of the five ethnic variables responded positively to transformation. Both the percent Hispanic and percent Asian American profited from natural log transformations.  Of the five socioeconomic variables no mathematical transformation was required.  The final set of variables containing the three social deviance variables had one variable, corporal punishment, which benefited from a square root transformation.  It was found that of the four subjective classification categories, three (EMR, SED, and Gifted) benefited from a natural log transformation, indicating that their original scale was not normal, but rather logarithmic.  The relatively simple analysis conducted so far tells us a number of important things about the data.  First, a number of the distributions were heavily skewed and would not be properly described by classical statistics in their original form, but rather needed to be rescaled.  It is interesting to note, for example, that in the original analysis of the two subjective categorizations (Gifted and SED), by Gelb and Mizokawa (1986), no correlations were observed.  It may be that the skew was not properly considered in the original analysis and therefore not properly assessed.  The table below is a revised correlation table of the variables we used in the subsequent analyses.

Revised Correlation Matrix

While there are several small shifts in the correlations between demographic variables and EMR and LD rates, the number of significant correlations with SED and Gifted categories increase from one to seven and zero to four respectively.  This is because the high skew in the original scale of these variables masked the strength of the relationship with these variables.

The two scatterplots below illustrate just how severe this case was in the original Gelb and Mizokawa data.  The scatterplots depict the relationship between the percent Asian American and Gifted variable.  The scatterplot to the left represents the original Gelb and Mizokawa data, while the one to the right presents the re-expressed data used by Dugan and Behrens.
 

Gelb and Mizokawa	Dugan and Behrens

 

We re-examined the four objective classification categories in relation to the newly transformed demographic variables and found that when contrasted against the original analysis conducted by Gelb and Mizokawa no substantive differences could be observed. Thus, no additional analyses were conducted upon the objective categories.

Regression Analyses
Having rescaled severely skewed variables, we turned next to the model building inherent in the regression analyses.   To lower the number of variables, Gelb and Mizokawa (1986) combined the standardized raw data of each variable and summed the variables together to form three aggregated variables of SES+ (personal income and education cost), SES- (infant mortality  rate, child poverty rate and public aid), and social deviance (corporal punishment rate and incarceration rate). While the standardization was an appropriate method to remove scaling level effects (numbers of different sizes), the adding together of the variables weighted them equally and ignored the fact that they may have contributed different amounts of information in the overall prediction.  To avoid this problem we choose to run principal components analyses on the three independent variables groupings (ethnicity, socioeconomic, and social deviance) to ascertain if a smaller set of factors could be used.

Principal Components Analysis
A principal components analysis of the five ethnic variables was first conducted. This initial analysis derived two components.   The two components were best described as either nonblack or nonwhite factors.  We used these factors in the regression analyses. A second principal components analysis was conducted with the socioeconomic variables. It was determined that only one factor was extracted from the component analysis when the aid variable was not included in the analysis. Thus, we used the single derived factor and the aid variable in the subsequent regression analyses. Our final principal component analysis of the social deviance variables produced one derived factor once the suspension variable was not included. This derived factor and the suspension variable were used thus in the regression analyses. The table below lists the final variables that we used in our regression analyses.
 

We first ran four multiple regressions using the six independent variables predicting each of our four dependent variables.  Unlike Gelb and  Mizokawa (1986), who used a backwards elimination method, we entered the all variables into the model.  We chose not to implement any form of forced model, such as forward or backward, due to our belief that the researcher was smarter than his or her computer and that it was important to check the model fit in terms of residuals and other regression diagnostics at each step of the variable elimination process.  This followed the general logic of a backward solution while minimizing the danger that the computations were fooled by anomalous outliers or other pathological patterns in the data. Thus, the final models were determined by way of judging how significant each variable was in the model along with the amount of variance accounted for by the regression equation given appropriate data patterns.  The following table presents the final regression models that were chosen in each case. Hyper-text with the regression-analysis source tables are provided through links on the names of the dependent variables. 
 

Final models (Coefficients are standardized, ordered by size)

LD = 4.075 -0.257 (aid) + 0.277 (nonwhite)

LEMR = 0.275 - 0.322 (nonblack) - 0.142 (nonwhite) -  0.170 (soc_d)

LSED = -1.529 - 0.419 (sespc) + 0.179 (suspend)

LGFTD = 0.202 + 0.287 (nonwhite)

 

In each case, all terms in the formula were significant at the 0.05 level.  The model quality was documented in plots of the raw and studentized residuals.  Examining these formulas provides a rather different account of the situation than obtained from the analysis by Gelb and Mizokawa (1986).   First, we found significant relationships between the rescaled SED and Gifted categorization rates and the SES principal component together with suspension rate and the nonwhite principal component respectively.  For each of these variables, the standardized regression coefficients were between 0.3 and 0.5.
 

In this document, we sought to reexamine the work of Gelb and Mizokawa (1986) by applying the detailed analysis procedures of EDA while taking advantage of the detailed reporting that is possible in a hypertext environment.   The EDA analysis revealed that Gelb and Mizokawa (1986) were somewhat misled by the skew and curvilinearity in their data because they relied primarily on classical statistics and mechanized inference of automated model selection.  Careful examination of the data and rescaling reveals that the case for their view is stronger than they had recognized, and that there are also strong correlations between demographic variables (particularly SES) and levels of SED and Gifted categorizations.  These relationships are, however, curvilinear in the original scale, and linear in the logarithmic scale.

With regard to the use of hypertext and hypermedia (graphics and supporting tables), we believe the use of such technology, as demonstrated here, can impact the standards for reporting research results.  Traditionally, statistical analyses and their presentation have been terse and often inadequate.  The traditional print format fails to allow for the direct distribution of datasets, the display of the numerous graphics that occur in exploratory data analysis, and the detailed reporting of statistical summaries such as have occurred here.  Such a detailed mode of presentation raises the bar for researchers since it encourages the reporting of all relevant aspects of the analysis and allows other researchers to (a)  examine aspects of the analysis not explicitly mentioned, and (b) check and reanalyze published work.

At present, the creation of detailed hypertext documents is not a simple task.  However, we expect that with the integration of World Wide Web standards in common word processors and statistical packages, such hypertext documents will be relatively easy to construct.  We encourage educational researchers to reconsider their traditional reporting (and analyzing) procedures given the new technologies of graphically intensive computing and hypertext document construction.

Behrens, J. T. (1997a). Principles and procedures of exploratory data analysis. Psychological Methods, 2, 131-160.

Behrens, J. T. (1997b). Does the white racial identity attitude scale measure racial identity? Journal of Counseling Psychology, 1, 3-12.

Behrens, J. T., & Dugan, J. G. (1996). A graphical tour of the White Racial Identity Attitude
Scale data in hypertext and VRML. Available at http://research.ed.asu.edu/reports/wrias.graphical.tour/graphtour.html

Behrens, J. T., Dugan, J. T., & Franz, S. (1997, August). Improving the reporting of research results using the World Wide Web. Paper presented at the 106th Annual Convention of the American Psychological Association, Chicago, IL.

Behrens, J. T., & Smith, M. L. (1996). Data and data analysis. In D. C. Berliner & R. C. Calfee (Eds.), Handbook of Educational Psychology, (pp. 949-989). New York: Macmillan.

Behrens, J. T., & Yu, Chong-Ho. (1995). Application of multivariate visualization to behavioral sciences. Behavior Research Methods, Instruments, & Computers, 27(2), 264-271.

Gelb, S. A., & Mizokawa, D. T. (1986). Special education and social structure: The commonality of "exceptionality." American Educational Research Journal, 23, 542-557.

Mosteller, D. E., & Tukey, J. W. (1977). Data analysis and regression: A second course in statistics. Reading, MA: Addison-Wesley.

Smith, M. L. (1982). How educators decide who is learning disabled. Springfield, IL: Charles C. Thomas.

Tukey, J. W. (1977). Exploratory Data Analysis. Reading, MA: Addison-Wesley.
 

Authors hold the copyright to articles published in Current Issues in Education. Requests to reprint CIE articles in other journals should be addressed to the author. Reprints should credit CIE as the original publisher and include the URL of the CIE publication. Permission is hereby granted to copy any article, provided CIE is credited and copies are not sold.

| home | articles | subscribe | submissions | editors | reviewers | | email: cie@asu.edu |

Copyright © 1998-2001 Current Issues in Education. All rights reserved. ISSN: 1099-839X