This file contains a description of important changes between different versions of S+Resample. The latest changes are at bottom. -------------------------------------------------- Changes between S+Resample 2.0 beta 1 and beta 2 (8/26/2002) ---- Changes that affect how you call functions ---- Added new function limits.percentile(), to replace limits.emp(). limits.emp() is deprecated. The name "empirical" is used for something else in the bootstrap literature. Summary functions no longer use the work "empirical" in the printouts. Added argument `subset.statistic' to plot.resamp() and qqnorm.resamp(), to allow plotting for only some components of multivariate statistics. Added argument `key' to plot.resamp(). Made the default to add a legend; use key=F for old behavior. Add symbols to help distinguish vertical lines at the observed and mean of the replicates. Histograms made by `plot.resamp()' no longer have blank space between bars; this is appropriate for a histogram of continuous data. To get the old behavior use `inside=T'. New versions of arrows() and segments() that: * make horizontal and vertical arrows and segments easier; do not need to specify the redundant dimension * have argument subset, to use only a subset of x1, y1, x2, y2 arguments. mean() now supports trim and weights simultaneously. skewness() and kurtosis() now support weights, using method="moment". Functions reweight(), tilt.after.bootstrap(), and limits.tilt() now include arguments `subjectDivide' and `modifiedStatistic'. bootstrap() now computes empirical influence values L by default if n is small enough; argument `L' has new options "choose" (in which bootstrap chooses the best method for computing L using the same criteria that resampGetL() does by default) and "none" (to prevent computation of L). This reflects the BootstrapNEW GUI as well. New function colTabulate(); fast tabulation by column. New function randomSample() -- similar to sample(), except samples rows of matrices, uses a faster algorithm, samples with unequal probabilities correctly, and supports sampling "with minimal replacement" when size>n or size*max(prob)>1. In the GUI for Linear Regression with Bootstrap, there are changes to the Bootstrap Tab: the "Subject" field has been removed (you can still specify this in the "Additional Arguments" field); the "Random Seed" field has been added; the "Correlation Matrix of Estimates" checkbox and "Save Summary Results As" field have been removed. The manual and GUI help file screen shots have not been updated to reflect these changes. The "Group" argument pull-down list is now automatically filled with the variables from the Model tab. ---- Other changes ---- plot functions fixed, so that if you plot a resample object as the first plot in your session, then do another plot, the first plot will be erased. permutationTest now works when some of the replicates are Infinity. If you call plot.resamp() and qqnorm.resamp() with axes=F, a box is no longer drawn. summary.bootstrap() now does bootstrap tilting intervals by default, if the statistic is suitable (if it is functional, and sample size is not too large or bootstrap object contains L). limits.tilt() now allows missing values, and omits columns of multivariate statistics for which the observed statistic is not inside the range of the data. The criteria for choosing the default method for computing influence values in resampGetL() has changed. The "ace" method is used more often (was using "ace" if n < 30 && sum(B) > 2*n+100, now it does if only sum(B) > 2*n+100). If "ace" is not used and there is stratified sampling, use "influence" only if the statistic can be modified to add weights, "jackknife" otherwise. bootstrap() now returns statistic.is.random as one component of the output (this is not documented in the help file, and may change in the future). bootstrap() now computes and returns `Lstar', the linear approximation to the statistic based on empirical influence values `L', if `L' is computed. samp.finite fixed to stop when N < size. Changed how `z0' is calculated in limits.bca() when any(replicates==observed); count equality as being half below. -------------------------------------------------- Changes between 2.0 beta 2 and beta 3 (11/21/2002) ---- Changes that affect how you call functions ---- Added group capability to a number of routines: -- revSaddlepointP() and revSaddlepointPSolve() for tilt = "exponential" -- tiltWeights(), tiltMean() and tiltMeanSolve() for "exponential" and "ml" (add new lambda parameterization). NOTE: the definition of exponential tilting weights has changed. See the ReleaseNotes or the help files for the tiltWeights() tiltMean() for details. The saddlepoint routines have not, however, kept pace with these changes. Use of the `group' variable with these routines (pSaddlepoint, saddlepointP, etc.) should be avoided until the discrepancy has been addressed. New function rquasi() for generating quasi-random numbers. New function limits.t (t-intervals with standard error from resampling). New functions permutationTest2() and bootstrap2() for comparing 2 samples. Add argument `full.partition' to samp.permute(), for generating complementary sets of indices. Add argument `observed.indices' to bootstrap(), to indicate which samples to use for computing the observed value of the statistic. stdev now accepts a `weights' argument. Add `xlim' argument to plot.resamp(), by default include observed value in `xlim'. Add a Bootstrap tab to the following GUIs: One-sample t Test; Summary Statistics; Correlations and Covariances; Proportions Test. Add a Bootstrap tab and a Permutations tab to the Two-sample t Test GUI. Add new argument `diffMeans' to permutationTestMeans(). If true (the default), then convert observed and replicates from treatment 1 sum to difference in treatment means. In bootstrap(), jackknife(), influence(), pbootstrap(), sbootstrap(), bootstrap2(), permutationTest(), permutationTest2(), permutationTestMeans(): * Added argument `label'; if supplied then a component `label' is included in the output, and used for printing and plotting. * A component `defaultLabel' is added to the output, and used for plotting (not printing) if label is not present. To avoid this, set label="", or main="" when plotting. * Added argument `statisticNames', if present then used as names for the dimensions of the statistic. Revert to old behavior--by default, L is not computed in bootstrap(), unless the statistic is mean or colMeans. print.resamp(), plot.resamp(), and qqnorm.resamp() look for a component `label' in the input object, use for labeling, and main title when plotting. If not present, then component `defaultLabel' is used when plotting. limits.t() has new arguments `df', `adjust', and `z'; used to determine whether to use t and normal quantiles, and what degrees of freedom. permutationTest2() and permutationTestMeans() now have arguments data2 (may supply instead of argument "treatment") paired (for paired permutation tests) The result now has a "label" component for better printing. The order of arguments for these functions has been changed for consistency. bootstrap2() has a new data2 argument; can now give two datasets as separate arguments, rather than combining them and specifying a treatment vector. If `L' is supplied as a numerical object, expect it to have the sign reversed for the second group. This makes the input value of `L' and the component `L' in the output consistent. ---- Other changes ---- bootstrap(), bootstrap2(), permutationTest2() handle simple mean and colMeans faster; the bootstrap functions also add component L to the output in these cases, to speed up later computations. Changes in terminology in GUI fields: change "Limits" to "Confidence Intervals"; change "empirical percentiles" to "percentiles" Clean up some parts of the graphical interface. Change "Empirical Percentiles" to "Percentiles", "BCa Limits" to "BCa Confidence Intervals", "Tilting Limits" to "Tilting Confidence Intervals". In bootstrap2, changed default B to 1000. Some technical fixes to ensure correct operation when data has 1 column. limits.tilt() now returns a matrix with attributes rather than a list. The attributes are the original list components (limits computed by two methods, numerical search parameters), and the matrix is the combined (most conservative) confidence interval. By default only the matrix is printed. limits.bca: centers L (in a bootstrap object) to mean 0 for each group and treatment before calculating acceleration. limits.bca and limits.t: handle a bootstrap2 object. -------------------------------------------------- Changes between 2.0 beta 3 and beta 4 (2/6/2003) Most important changes: * speed, for some operations * rewritten graphical interface ---- Changes that affect how you call functions ---- Change the default for t.test to var.equal=F, and change the name of the test that assumes equal variances from "standard" to "equal-variance". This change reflects best current statistical practice. Little is lost by not assuming equal variances if the variances are in fact equal, but assuming equal variances when they are not gives poor results in general. Change function names: dSaddlepoint to dDiscreteMean pSaddlepoint to pDiscreteMean qSaddlepoint to qDiscreteMean Change argument name in those functions, for consistency with [pdqr]discrete: L to values In two-sample and other multiple-group problems, saddlepoint calculations can be done for the sum of sample means, rather than only for an overall mean. Add argument `mean' to saddlepointP saddlepointPSolve saddlepointD New resampleOption(), provides options controlling printing. New function print.bootstrap2, adds option to print summaries for the two groups. ---- Other changes ---- Bootstrapping a sample mean is now much faster. Bootstrap tilting is now much faster, due to resampling indices being saved (by default for small and medium runs). This uses a new capability for storing compressed indices. The graphical interface (Windows only) has been substantially rewritten. The bootstrap tabs added to various dialogs are now consistent. The GUI calls new routines (e.g. bootstrap2), and takes advantage of new capabilities in existing functions. In jack.after.bootstrap and related functions, change the name "relative influence" to "standardized influence", and define it. The plot has a new y label. Change t.test, for better compatibility with version in S-PLUS. Now adds an attribute to the usual "htest" object, that contains tilting inferences (now called saddlepoint, to avoid confusion with tilting implemented by resampling). New name for default bootstrap sampler, is samp.bootstrap. -------------------------------------------------- Changes between 2.0 beta 4 and beta 5 (12/15/2003) The major change is additional work on the graphical interface (Windows only), including: * revised and new menus under Statistics:Resample * pressing the Help button on GUI menus leads to appropriate help, * the context menu for bootstrap is revised (i.e. what happens when you click on a bootstrap object from the Object Explorer). This work is only partially complete. samp.permute now accepts a size argument. This allows for half-sampling, for example. Improve handling of label and defaultLabel components in some cases. New function resampleOptions(), controls some printing, including option for print.resamp to omit printing the call. Maximum likelihood tilting now supports stratified sampling and multiple-sample problems, including bootstrap2. limits.tilt now supports bootstrap2 objects. Turn off warnings about conflicting definitions (unfortunately, this also turns off warnings that are appropriate, if someone inadvertently masks something). Specifying limits.bca(..., acceleration = 0) now gives BC (bias-corrected) intervals when there are multiple statistics. -------------------------------------------------- Changes between 2.0 beta 5 and beta 6 (8/4/2004) Most important changes: * Add support for bootstrapping ratios (of 2 means, 2 variances, ...) both in the command line and GUI. * Easier way to permute only certain variables for permutation testing. * Add permutation testing for proportions and correlations in GUI. * Add trimmed means to 1 and 2-sample t-tests and Summary Statistics. * More extensive cross references in help files. New `resampleOverview' help file is an annotated list of cross references. ---- Changes that affect how you call functions ---- Add argument `ratio' to bootstrap2(), permutationTest2(), and permutationTestMeans(). If TRUE, then resample the ratio of statistics between two samples, rather than difference. Add support for ratios in functions resampMakeLabel(), resampGetL.bootstrap2(), and limits.tilt(). Add argument `resampleColumns' to bootstrap(). If supplied, then resample only those columns. This is useful for permutation testing. cdf() and plotCDF() are now generic functions, with resamp methods. You can now do e.g. cdf(bootstrap object, q) plotCDF(bootstrap object) groupMeans(a data frame, group, repeats=T) now works. Add new function scatterPlot(), to simplify getting a scatterplot for multiple columns of a resample object (note that "plot.resamp" does histograms, not scatterplots). Add pairs.resamp() to simplify getting pairs plots for a resample object. resampleOptions has new options, indicating whether certain confidence limits should be produced by summary.bootstrap and summary.resamp ---- Other changes ---- In t.test, allow nx=1 or ny=1 when var.equal=T. Modified t.test to use different lower bounds for saddlepoint p-values. t.test: Use label "Pooled-Variance..." rather than "Standard...", to avoid implying that pooled-variance tests and confidence intervals are more respectable than the non-pooled versions. Modified dispatching for mean, so that bootstrap(a data frame, mean) works (however, it is better to use colMeans than mean for a data frame). Similarly modified dispatching for cor, var, quantile, median. Removed a statement in .First.lib that causes problems for emacs ESS. Add argument "label" to pbootTest, bootPred, crossVal. Make the corresponding print methods consistent with print.resamp, in use of argument printCall, and components call, label, and defaultLabel. Add `resampleColumns' to resampCor() and resampVar(). Use in correlations GUI for permutation testing of correlations. Error checking, bootstrap2() and permutationTest2() check that `group' argument is the right length. bootstrap() sets `group.order.matters' appropriately if `resampleColumns' is supplied, or if sampling with modified size. permutationTest2() has improved calls to bootstrap when `data2' is supplied -- includes expressions like `1:n1' symbolically rather than giving a long vector of data. permutationTest and permutationTest2: Do not compute linear approximation `L' by default. ---- Graphical Interface (Windows) ---- Add copies of six "augmented menus" (add resampling tabs to existing statistics menus) under "Statistics : Resample" for easier access. Add permutation testing to GUIs for proportions and correlations. Resample menus no longer overwrite some existing menus under Statistics. In proportions menu, do stratified bootstrapping when there are multiple proportions. Two-sample t-test menu, calls to bootstrap are now shorter in some cases, pass data symbolically rather than numerically. Two-sample t-test menu, fix inconsistency between t.test results and bootstrap results. Both now define the "first" group as the one that occurs first in a grouping variable (rather than converting to factor and taking lowest level). Two-sample t-test, when select "paired", permutationTest is now enabled, and does appropriate test. When variable2 is a grouping variable, do not disable "paired". -------------------------------------------------- Changes between 2.0 beta 6 and beta 7 (1/20/2005) If you forget to attach the resample library in first position, it is moved for you. limits.bca() now works for a bootstrap object created using sampling by both group and subject. If limits.bca() tries to call jackknife and fails (typically because variable lengths don't match up), it now returns NULL rather than failing. As a result, summary(a bootstrap object) omits BCa limits rather than failing. Some functions are now generic, with methods for resamp objects: density quantile rowMeans, rowSums, rowVars, colMeans, colSums, colVars For example, if `boot' is a bootstrap object, you can now do density(boot) instead of density(boot$replicates). The methods use weights if they are present. Some functions now accept a freq argument (frequencies), including density quantile rowMeans, rowSums, rowVars, colMeans, colSums, colVars rowMeans and similar functions have a new `n' argument, which makes these functions behave like fastRowMeans and similar functions. t.test now accepts a treatment argument; this makes it easier to do two-sample t-tests when both groups are in a single data frame. Can now plot bootstrap results when 25% & 75% percentiles are the same. pbootstrap (and sbootstrap) summary statistics now use the same order as bootstrap: Mean before Bias. bootPred(..., assign.frame1=T) now removes the copy on frame 1 before exiting. Tests in loop/ now remove objects they create. ---- Graphical Interface (Windows) ---- Fix problem with running AOV from menus (specific to S-PLUS 6.2). From two-sample t menu, printouts now use labels of groups rather than numerical codes. Permutation test menu, fix problem when switching back and forth between two samples or selecting columns to permute. -------------------------------------------------- Changes between 2.0 beta 7 and beta 8 (4/14/2005) This version assumes S-PLUS 7.0 or later. Changed some generic functions for compatibility with S-PLUS 7.0. Removed functions: anyMissing() notSorted() These are now part of S-PLUS 7.0. plot.resamp() now works when x$estimate$Mean is NULL Added help file "tiltDetails". Add permutations() and samp.permutations() (all possible permutations). In the "Permutation Test" menu, the Strata/Groups, Subject, Random Seed, and Assign Resample Data to Frame 1 menu choices work. -------------------------------------------------- Changes between 2.0 beta 8 and beta 9 (11/11/2005) Fix error in cor(x, y, na.method = "available") Fix minor error in print.abc() Unix only: package S-PLUS objects as a single file .Data/__Objects -------------------------------------------------- Changes between S+Resample 2.0 beta 9 and S+Resample 3.0 (4/27/2007) This version is only for S+7.0.6 and later. ---- Changes that affect how you call functions ---- Have removed some functions that are in S+7.0.6 including: colMaxs colMins colRanges rowMaxs rowMins rowRanges colMeans colSums colVars rowMeans rowSums rowVars allTrue expectWarnings hist (generic function and S4 default) nclass.fd arrows.default segments.default Have removed function previously deprecated: fastColSums fastColMeans fastColVars fastColStdevs fastRowSums fastRowMeans fastRowVars fastRowStdevs (instead use n argument to colSums or similar functions) Have removed additional functions resamp.set.seed (and argument sampler.setup to bootstrap) resamp.return.seed (and argument sampler.wrapup to bootstrap) rquasi (to appear in a separate lowDiscrepancy package) Some function names are changed to be more informative: -- old -- -- new -- bootPred bootstrapValidation bootstats makeBootstrap bootstrapMeans indexMeans bootstrapProducts indexProducts bootstrapSums indexSums bootstrapVars indexVars crossVal crossValidation jack.after.bootstrap jackknifeAfterBootstrap pbootTest parametricBootstrapTest pbootstats makeParametricBootstrap pbootstrap parametricBootstrap sbootstrap smoothedBootstrap tilt.after.bootstrap tiltAfterBootstrap The old functions still exist but are deprecated - they will give a warning once per session, then call the new versions. Additional functions are deprecated; see help(Deprecated.resample). The "abc" class (created by limits.bca) is now "limits.abc". Functions resamp.set.seed and resamp.return.seed are removed, as are the corresponding arguments sampler.setup and sampler.wrapup to the bootstrap function. concomitants and controlVariates are now generic functions, with * a default method, and * a bootstrap method. The default methods are general-purpose functions, used for adjusting the distribution of a variable y based on a covariate x that has a known distribution function (for concomitants) or known mean (for controlVariates). The bootstrap methods are for after-the-fact adjustment of a bootstrap distribution, and return modified bootstrap objects. See help(concomitants) help(concomitants.bootstrap) help(controlVariates) help(controlVariates.bootstrap) Backward compatibility note -- concomitants.bootstrap replaces the old concomitants function. The old "concomitants" class is deprecated -- you can change the class to "concomitants.bootstra" using e.g. oldClass(object) <- c("concomitants.bootstrap", "bootstrap", "resamp") Removed log argument from bootstrapProducts (this was intended for speed, but was actually slower). Major revision on jackknifeAfterBootstrap (formerly jack.after.bootstrap). There is a fundamental problem with the analytical bias and standard error calculations that the previous version did by default -- they were too sensitive to random variation caused by a bootstrap sample size less than infinity (they would be fine if your computer were infinitely fast). The new version defaults to a percentile-based graphical approach instead. It also incorporates variance-reduction techniques for both approaches; this partially mitigates the fundamental problem. Additional plotting capabilities, and the default plot is easier to understand. Major revision of tiltAfterBootstrap (formerly tilt.after.bootstrap). This also has different defaults and plots, that parallel the jackknifeAfterBootstrap changes. The order and names of arguments for ddiscrete, pdiscrete, qdiscrete, rdiscrete has been changed for better consistency with certain functions outside the resample library. New function samp.half, for half-sampling. For bootstrap2, add an argument twoSample.args, to support passing different arguments to bootstrap for each of the two data sets. ---- Other changes ---- Fix minor error in print.abc Unix only: package S-PLUS objects as a single file .Data/__Objects Add new argument 'njack' to samp.bootknife. Make limits.tilt handle a concomitants object. indexMeans indexProducts indexSums indexVars now support logical and character matrices of indices. limits.bca now returns NAs instead of failing if some statististic is all NA. limits.tilt no longer omits rows with NAs in any statistic (so NAs in one statistic won't bias the intervals for another statistic). Faster versions of mean, median, groupMeans and other group* functions. This is only for S+8.0 and later; if you are using an earlier version old copies of these functions will be used. Added "freq" (frequencies) argument to a number of functions, generally those with weights arguments. There are two overview help files -- one for the command-line, and one for the graphical interface (Windows only). There are now links or notes pointing back and forth between these files. bootstrapValidation (was bootPred) now allows missing values. A bug in the computation of the .632 and .632+ estimates when sampling by subjects is fixed. It warns when the number of bootstrap samples is too small to provide accurate estimates. combinations and permutations return integer arrays (previously double precision in odd cases like n=0). permutations now supports permutations of size smaller than n. When resampling residuals for linear regression, a flag is set that prevents the accidental use of the bootstrap results in downstream functions that an not appropriate when resampling residuals. There is a new variation of resampling residuals, the wild bootstrap. See help(bootstrap.lm) addSamples now supports weights summary.bootstrap how continues if limits.tilt or limits.bca cannot be calculated. For bootstrap2, fix the computation of L (was incorrect in some cases). ---- Graphical Interface (Windows) ---- The help buttons from resample menus now work for all menus. For the linear regression/resample menu, there are new built-in options for resampling predictions or regression lines. The GUI now also allows "graphical bootstrapping", when the statistic has length zero (which is the case when just drawing regression lines). There are additional context menus for additional classes of objects. Not all classes of resample objects show up by default in the Object Explorer. You can add them using the new function updateObjectExplorerClasses. Added menu items to open tutorial.ssc and resampleExamples.ssc