TwelveByTwelve (TBT)      Pilot Study      Explicit Transformational Syllabus (XTS)     Fundamentals of TwelveByTwelve

Pilot Study
by Luby Prytulak, PhD
You are at
Email comments to:
First posted  09 May 2012 08:15pm,  last edited  19 Apr 2013 02:48pm

TwelveByTwelve (TBT) is a teaching method which removes obstacles to students completing a Grade 12 curriculum by the age of 12, with the following qualifications:

A pilot study testing an early version of TBT involved three students and produced encouraging results, a small sample of which is presented below.

Anything that any one of the three students is shown doing below could also be done by the other two, and at a comparable level of mastery, from which arises the hypothesis that nine months of study under TBT raises a student to approximately the same high level of achievement whether the student has been working while nominally in Grade 6 (Kirsten), or Grade 5 (Alexa), or even Grade 2 (Marko).  The sole exception to this rule is Kirsten's masterful drawing of the bones of the hand — all three students did draw daily, and over nine months each produced a portfolio of work, but Kirsten had entered TBT with strong artistic talent already established, and only she chose to add to her portfolio a work that required prolonged attention.

The numerical entries in the table below have the format YY:MM, such that Marko being seven years and nine months old at the time of video recording is shown as 07:09, his having at that time received one academic year of conventional schooling is shown as 01:00, his having received nine months of TBT schooling is shown as 00:09, and his total schooling being the sum of the two, which is one academic year plus nine months, is shown as 01:09.

MARKO 07:09 01:00 00:09 01:09
ALEXA 11:01 04:00 00:09 04:09
KIRSTEN 12:01 05:00 00:09 05:09

A video introduction to the three TBT students in non-academic settings.

TwelveByTwelve (TBT): Three Students in Hot tub
  TwelveByTwelve (TBT): Grouse Mountain outing
  TwelveByTwelve (TBT): Horseback riding

As mathematical competence is currently a topic of public interest, a comparison of TBT and conventional students on a few mathematical tasks might be a good place to begin.  It must be understood that the six math decks shown below are only a fraction of the math decks that the students worked with, and that math decks were only a fraction of their math activities, a more typical activity being solving Canadian Mathematics Competition problems sitting together around a table.  Math decks were selected for filming because they showed rapid activity, whereas filming problem-solving would have shown students mostly hunched over their work, occasionally jotting something down on their notepads, and entering into discussion only infrequently.

Piano sometimes audible in the background is one of the students currently not on video either daily practicing alone or receiving weekly instruction.

Marko in TwelveByTwelve (TBT)
Age 07:09   Education 01:09
  TwelveByTwelve (TBT): Marko does PRIME FACTORS
  TwelveByTwelve (TBT): Marko does GET THAT ANGLE

Alexa in TwelveByTwelve (TBT)
Age 11:01   Education 04:09
  TwelveByTwelve (TBT): Alexa does SOLVE FOR X
  TwelveByTwelve (TBT): Alexa does SOLVE FOR WHATEVER

KIRSTEN in TwelveByTwelve (TBT)
Age 12:01   Education 05:09
  TwelveByTwelve (TBT): Kirsten does GET THAT AREA
  TwelveByTwelve (TBT): Kirsten does GET THAT EQUATION

Mathematics is far from the only subject taught in TBT, and in fact it is fair to say that everything that is taught in conventional school is also taught in TBT, though at a much higher level, and a great many things are taught in TBT that are never taught in conventional schools, at least not to students of the same ages as the three TBT students.  Below is a small sample of the many things taught in TBT that are not often taught elsewhere.

The KIRSTEN DRAWS HAND video below calls for a word of explanation.  According to a roughly-outlined script, Marko was supposed to ask Kirsten whether she was almost finished her drawing, and she was supposed to say that she was just touching up some particular bone, and Marko was then supposed to ask whether she could name that bone, and when she did name it, he was supposed to ask whether she could name all the bones, and she was supposed to demonstrate that she could.  However, both students sensed the artificiality of this proposed script, and for that reason veered off in their own direction.  Kirsten started by telling Marko, who was unsuccessfully trying to stifle his laughter, "Don't breathe so loud!" upon which the two became so convulsed with laughter that Marko said what sounded like "Your face is red with anger", though he must have meant "You face is red with laughter".  By the time that Marko tried to begin following the script by asking "Are you almost finished?" Kirsten was laughing too hard to be able to answer, though she did eventually manage to say "Yeah, I'm just doing this one", to which Marko said "Can you name it?" to which Kirsten said "No, I'm too dumb" to which Marko asked "Can you name any one?" to which Kirsten said "Sure, I can name it", and pointed to the labelled illustration in the book open in front of her.  You have to watch the video to understand why the laughter is infectious.

To illustrate the comparability of achievement, the KIRSTEN DRAWS HAND video shows Kirsten naming the 29 articulated foot bones in a leisurely 0:14, which is to say, in fourteen seconds.  In speed tests throughout preceding months, however, the personal-best record times were Kirsten 0:06, Alexa 0:07, and Marko 0:08.  That is correct — six seconds is the record time for naming and pointing to the 29 foot bones, a speed made possible by such expedients as naming the five metatarsals by saying "First to fifth metatarsals" and indicating them with a sweep of the pointer.  In setting records, a time would be credited only if every syllable of the recitation had been clearly audible.

Speed is valued in TBT for three reasons: (1) the practical application of the skills in a career would demand speed, (2) a lot of practice gets done in the available time, and (3) students love speed.  The table below summarizes some times that can be watched by clicking on the images below, as well as some personal bests accumulated during the preceding months but that had not been video recorded.

Identifying 62 features of the HUMAN SKULL, and identifying 54 WOOD SPECIES, are shown on the video as group activities, whereas normally a student would run through the identifications in a set order and alone, which is to say with only a monitor present to handle the stopwatch and verify that all identifications were being made, which solo performance made it possible for the student to set personal bests.

Task  Watchable On Video   Kirsten 
personal best
personal best
personal best
Identifying 29 articulated footbones  Kirsten   0:14 0:06 0:07 0:08
Identifying and assembling 29 jumbled footbones  Kirsten   1:41 1:03 2:17 1:54
Assembling five lumbars blind Alexa   0:40 0:29 0:26 0:32
Identifying 62 features of the skull group activity 0:45 0:52 1:18
Identifying 54 wood species group activity 1:22 1:00 1:51

TwelveByTwelve (TBT): Kirsten draws hand and assembles foot
Kirsten draws hand
and assembles foot
  TwelveByTwelve (TBT): Alexa assembles lumbars blind
Alexa assembles
lumbars blind
  TwelveByTwelve (TBT): Kirsten, Alexa, and Marko identify features of the human skull
Identifying features
of the human skull
  TwelveByTwelve (TBT): Kirsten, Alexa, and Marko do bones on the floor
Bones on floor

TwelveByTwelve (TBT): Kirsten plays piano
plays piano
  TwelveByTwelve (TBT): Reading Pitman Shorthand
Pitman Shorthand
  TwelveByTwelve (TBT): Kirsten writes Pitman Shorthand
Kirsten writes
Pitman Shorthand

TwelveByTwelve (TBT): Alexa identifies verb tenses
Alexa identifies
verb tenses
  TwelveByTwelve (TBT): Marko does French
does French
  TwelveByTwelve (TBT): Kirsten, Alexa, and Marko do Heart and Blood
Heart and Blood

In the ANALYSIS OF BEHAVIOR video below, it can be seen that in order to avoid blocking the view from the camera, Marko positions himself far back from the book, which makes it harder for him to see what he's reading, and easier to lose his place.

IDENTIFYING WOOD SPECIES and IDENTIFYING DOG BREEDS, which would be derogated in conventional schools as "rote learning", are valued in TBT because of their contribution to perceptual awakening — in order to learn to name, the student must begin to make fine discriminations, which is to say must begin to notice subtle characteristics that she has never noticed before, and which is to say must begin to see where before she was blind.  The perceptual awakening becomes evident throughout the student's life.  For example, to a conventional-school student, the various woods that are on display in a furniture store, or in an antique shop, tend to be beneath notice, whereas a TBT student examines them with interest, identifies them with a warm sense of recognition, discusses them.  Similarly, every dog the student sees in the street triggers an identification accompanied by a sense of joy, and when a rare breed is encountered, accompanied by delight.

Also reported in the above table are personal bests for identifying 54 WOOD SPECIES (the blocks being arranged not as in the video, but rather stacked in haphazard order in maybe four piles, and the student picking each block up and placing it aside as she named it).

In the recall phase of the MNEMONICS DEMO, the students' real names had been called out by the monitor, Alexa, but were erased from the audio track of the video.  In one instance the erasure accidentally extended too far, such that when Alexa says "twenty-four", only the "four" remains audible.

TwelveByTwelve (TBT): Kirsten, Alexa, and Marko study Skinner's ANALYSIS OF BEHAVIOR
Analysis of Behavior
  TwelveByTwelve (TBT): Kirsten, Alexa, and Marko identify wood samples
Identifying wood species
  TwelveByTwelve (TBT): Kirsten, Alexa, and Marko identify dog breeds
Identifying dog breeds
  TwelveByTwelve (TBT): Kirsten, Alexa, and Marko put on a mnemonics demonstration
Mnemonics demo

Declamation or recitation is another activity that conventional schooling bans as "rote learning".  In TBT, however, declamation is valued as an essential daily exercise not only for the lessons that its content teaches, but also for its development of language skills.  As always, whatever one student is shown reciting below can be recited by the other two as well.

TwelveByTwelve (TBT): Kirsten declaims TALK IS BY FAR
Kirsten declaims
Talk is by far the most accessible
  TwelveByTwelve (TBT): Kirsten declaims IF YOU WILL NOT FIGHT
Kirsten declaims
If you will not fight for right
  TwelveByTwelve (TBT): Alexa declaims THE DISCIPLINE OF THE SHIP
Alexa declaims
The discipline of the ship requires
  TwelveByTwelve (TBT): Alexa declaims ATTENTION TO DETAIL
Alexa declaims
In the end it is attention to detail

TwelveByTwelve (TBT): Marko declaims PERHAPS THE MOST VALUABLE
Marko declaims
Perhaps the most valuable result
  TwelveByTwelve (TBT): Marko declaimes THE FOX AND THE CAT
Marko declaims
The fox and the cat as they
  TwelveByTwelve (TBT): Marko declaims EVEN MY SHORT EXPERIENCE WITH WAR
Marko declaims
Even my short experience with
  TwelveByTwelve (TBT): Marko declaims AS FAR AS MY EXPERIENCE GOES
Marko declaims
As far as my experience goes

Are the TwelveByTwelve standards realizable?

The table below summarizes those of Marko's achievements by the age of 12 that come with evidentiary support.  In every category but Music and Art, Marko exceeds the TBT standard by showing competence at the first-year-university level instead of at the TBT-mandated Grade 12 level.  The links in the table take the reader to documentation.

The achievements shown in the table fall into two categories.  The first category is achievements that are so rare that they may have been equalled by only a handful of students in all of the United States and Canada, and in some cases may be so rare that they were quite unequalled anywhere.  In contrast, the last four lines in the table below document a lesser level of accomplishment, a level that is not only equalled but surpassed by vast numbers of twelve-year-olds, perhaps even by tens of thousands of them in the United States and Canada, or even hundreds of thousands.  For example, there are obviously vast numbers of youngsters whose French is of a higher caliber than Marko's, perhaps because they have immigrated from a French-speaking country, or have been raised in a French-speaking family, or have been educated in French-immersion schools.  The same with Ukrainian.  And vast numbers of twelve-year-olds undoubtedly can play better piano, or other musical instrument.  And vast numbers might evince superior artistic skills.

But such comparisons are inappropriate because TBT does not claim to raise twelve-year-olds to prodigious achievement in every area, but only to achievement that is markedly above the average of all twelfth-graders in every area.  To consider a concrete example of what this implies — if there are some 3,000,000 twelfth-graders in the US, then for twelve-year-old Mabel to have music skills that place her in the 90th percentile among them is also to have some 300,000 of them demonstrating music skills superior to hers.  Getting into the 90th percentile, even among students five years her senior, will not get Mabel on America's Got Talent.  But to have an educational system that places all twelve-year-olds in the 90th percentile compared to all twelfth-graders in every subject is to have the best educational system in the world.  That such a system will incidentally produce some prodigious achievement may be hoped for, but is not claimed, and should not be held out as a criterion of TBT success.

Another evaluation of TBT effectiveness may be to ask, Where are they now?  It would be fair to make a Where are they now? comparison between graduates of TBT and of conventional schooling, say one and five and ten and twenty years down the road, and on any dimensions at all, let's say on math and verbal skills to compare how smart they are, and on income to compare how they're faring in the world.  And everybody who has studied statistics and scientific method and has done behavioral research can predict with confidence what such a comparison would find — the TBT graduates will initially score much higher than conventional graduates on all three dimensions, but both groups will regress toward the mean over time, meaning they will all become increasingly average, and so the difference between them will shrink.  Half a century after graduation, it will be hard to distinguish one group from the other — time and chance will have worn away at them all, and with each passing year, the quality of education they had received in youth will make less and less difference.  That is the way the world works.  But as it is a given that differences between the two groups will shrink over time, is Where are they now? a good way to compare the two educational programs?  If differences are sought, why not measure them when they are fresh and large, why wait to measure them as they are in the process of vanishing?  And would it not be terribly unfair to simply follow the TBT group after their graduation, and point to their regression toward the mean as evidence of the failure of TBT?  Unfair because it would be omitting to notice that the conventional group had regressed also, and that the conventional group continued to underperform the TBT group.

TBT claims to produce startling educational effects while it is in operation; it does not claim to have conquered the phenomenon of regression toward the mean.  If after leaving TBT the student fails to thrive in the conventional classroom, then a fitting summary is to point out that that student had started out failing to thrive in conventional schooling, began to acquire excellence under TBT, and began to regress toward failing to thrive again when returned to conventional schooling.  That conventional schooling is able to dismantle some, and eventually a lot, of what has been built by TBT stands to the discredit of conventional schooling and not of TBT, and serves rather to highlight the value of TBT.  Were TBT methodology employed throughout an academic career, let us say to the very end of graduate or professional school, then its benefits would not be dismantled by exposure to conventional schooling, but rather would continue to accrue and compound, and the high skills available upon ultimately entering the work force could be applied to the service of mankind.

To put it another way, excellence is produced only by certain rare environments.  Departure from those rare environments is inevitably followed by a progressive deterioration of excellence.  That TBT excellence is fleeting is only a particular manifestation of the rule that all excellence is fleeting.  Fleeting excellence is worth pursuing because it is the only kind of excellence there is.  Fleeting excellence can be made less fleeting by expanding and proliferating the environments which foster it.

UBC Math 140 Differential Calculus for Arts students   10:10 74%
UBC Math 141 Integral Calculus for Arts students   11:11 85%
UBC Math 100 Differential Calculus for Science students   12:03 84%
UBC Math 101 Integral Calculus for Science students   12:07 95%
UBC Physics 110 First-year university Physics   12:07 80%
UBC Chemistry 103 First-year university Chemistry   10:07 80%
UBC Computer Science 124 First-year university Computer Science   12:10 80%
UBC French 110 First-year university French   12:11 83%
UBC Ukrainian 325 Level II Ukrainian   10:07 76%
Toronto Conservatory Grade 6   11:04 est 87%
Portfolio Selections 12 n.a.

What's missing above?  English, History, Biology, Phys Ed.  Within the pre-teen years in which we are interested, much was accomplished in these areas as well, but they were not studied formally at UBC until after age 12.  The only remaining standardized-examination-quality proof of achievement is Marko's Stanford-Binet and Kirsten's and Alexa's Canadian Achievement Tests which are helpful at this point mainly in reflecting on the three students' language skills, and which incidentally serve to reinforce the positive impression of their mathematical skills.

TwelveByTwelve (TBT): Marko hikes to mountain peak

TwelveByTwelve (TBT): Marko does backstroke    TwelveByTwelve (TBT): Marko skates

Marko's Stanford-Binet

Evidence of Marko's language skills comes from the Stanford-Binet Intelligence Scale: Fourth Edition that British Columbia education authorities administered when he was 10:06, and from whose five-page report the uppermost five scores in the excerpt below are relevant, indicating as they do that TBT had not been neglecting the development of language skills.

TwelveByTwelve (TBT): Marko's Stanford-Binet results

Marko's Stanford-Binet scores are understood here to be measures of the achievement that had been produced by TBT methodology, serving simply to reinforce what the array of UBC exams has been saying above.  As TBT students participated in a very broad assortment of intellectual exercises, one may conjecture that they were ready for anything that could be thrown at them.  Almost anything, as Marko did receive a single "average" score, the 60th percentile for "pattern analysis", whatever that is.

The alternative interpretation that Marko's performance, whether academic or Stanford-Binet, is indicative of innate giftedness is not borne out by the three most pertinent observations:

(1)  Marko's parents' academic records bear no resemblance to his.

(2)  During conventional schooling in Kindergarten and Grade 1, Marko was never singled out by teachers as being gifted or precocious or a model student, and neither were Kirsten or Alexa during their longer conventional-schooling exposures.

(3)  During the two consecutive academic years that Kirsten and Alexa were in TBT along with Marko (only the first year of which was sampled on video), their achievements tended to exceed his (as was to be expected from their being older), from which it is reasonable to expect that they would have continued to outpace him in the years to come had resources permitted the TBT pilot study to continue including them, and that they would have accumulated the same testimonials of university-level learning that Marko accumulated.

The evidence, then, points to the conclusion that the indications of superior performance discussed on this page can be credited to TBT methodology, which future publications will disclose, and cannot be credited to innate or inherent or genetic giftedness.

Kirsten's and Alexa's Canadian Achievement Tests (CAT)

After the first nine months of TwelveByTwelve, British Columbia educational authorities measured Kirsten and Alexa on the CAT, the key results being reproduced below, and a quick glance at which gives the impression of admirable performance.  One highlight is that both girls scored in the 99th percentile on all three of the Mathematics scales on their reports: MATHEMATICS COMPUTATION, MATH CONCEPTS & APPLICATIONS, and MATH TOTAL.  Perhaps if the CAT gave out the percentile rank to two decimal places, Kirsten and Alexa would have registered 99.99 the way Marko did on all three of his QUANTITATIVE REASONING scores on his Stanford-Binet.

And the two highest peaks were as follows.

Kirsten, nominally in Grade 6.8, performs at the Grade Equivalent of 12.6 in MATH CONCEPTS & APPLICATIONS.  As we can see on her report that a Grade Equivalent of 10.2 got her into the 99th percentile on another scale, then this Grade Equivalent of 12.6 may well have placed her into the 99.99th percentile in MATH CONCEPTS & APPLICATIONS if calculation to two decimal places had been allowed.

And Alexa, nominally in Grade 5.8, performs at the Grade Equivalent of 12.2 in LANGUAGE EXPRESSION.  And as we can see on her report that a Grade Equivalent of 9.2 got her into the 99th percentile on another scale, then this Grade Equivalent of 12.2 may well have put her into the 99.99th percentile in LANGUAGE EXPRESSION if calculation to two decimal places had been allowed.

Kirsten summary of results from her Canadian Achievement Tests


Alexa summary of results from her Canadian Achievement Tests


Let's look now for weaknesses.  Kirsten scores abysmally on LANGUAGE MECHANICS, and which pulls her TOTAL LANGUAGE down to barely above average.  LANGUAGE MECHANICS also happens to be Alexa's lowest score, suggesting that the deficit is less personal than caused by TwelveByTwelve neglect of that area of study.  The girls' detailed reports show that of the five topics which comprise LANGUAGE MECHANICS, the two lowest scores for both girls were awarded for COMMAS and BEGINNING WORDS/TITLES.  Admittedly, during the first nine months of TwelveByTwelve studies, commas had received no coverage whatever.  As for BEGINNING WORDS/TITLES — I cannot imagine what that skill might consist of, and wonder whether it deserves a place on any list of fundamental skills.

REFERENCE SKILLS is the next-lowest score in each of the two above CAT reports, and my examination of the detailed results indicates that if TwelveByTwelve had taken a few minutes to point out to its students the features of a LIBRARY CATALOGUE CARD, and of a DICTIONARY PAGE, then the REFERENCE SKILLS score would have risen to the level of the girls' stronger performances.

The CAT weaknesses documented above, then, lie in four areas that had been omitted during the first nine months of TwelveByTwelve study, but three of which measure skills which seem to be neither fundamental nor demanding: BEGINNING WORDS/TITLES, LIBRARY CATALOGUE CARD, and DICTIONARY PAGE, and the fourth of which, COMMAS, might be brought up to CAT 99th percentile level by, say, reading Strunk and White's pages 1 through 7 (in the 1959 edition that I happen to find at hand), followed by some drill, both of which might be managed within a couple of hours.

Two considerations must be kept in mind while evaluating the above CAT scores.

First, the stellar performances are not the result of a lifetime of TwelveByTwelve study; they are the result of only nine months of TwelveByTwelve study following comparatively unproductive years of conventional schooling — five in Kirsten's case, four in Alexa's, and one in Marko's.  Had TwelveByTwelve training been practiced throughout each of the three academic careers, performance would have been proportionately better.

Second, the CAT is biased in favor of students in conventional schools.  First of all, it tests only the conventional curriculum, which of course places students following any other curriculum at a disadvantage:

Educators told us they wanted tests that matched their curriculum.

The test specifications were drawn up in view of current Canadian curricula.

And teachers in conventional schools also have access to materials designed for the sole purpose of raising CAT scores:

The Teacher Resource Manual offers more than 500 instructional activities that have been carefully matched to CAT·3 objectives.

And copies of the CAT have not only passed through the hands of innumerable teachers, but can be freely purchased by a long list of other people:

With respect to security gaps that creators of recycled exams are obligated to attempt to plug, CAT creators seem naive, as for example with respect to who may be allowed to supervise a student's test taking, and who may be allowed to handle a student's answer card, and whether unauthorized copies of the CAT can be expected to leak out through security gaps and be put into illicit circulation.  An example of all three issues being handled negligently is CAT manufacturers warning homeschoolers who have purchased the CAT for their own children as follows:

After testing, answer documents must be returned for scoring along with all test materials within 30 days after they have been received.

Therefore, students in conventional schools, and some others, appear to have an immense advantage over students like the TwelveByTwelve duo whose teacher had not known that they would be tested on the CAT, and being unconnected to conventional education had never even heard of the CAT.  The result: the CAT tested COMMAS which TBT students had not studied, but did not test the Lewis diagrams distinguishing 1-Propanol from 2-Propanol which they had; the CAT tested LIBRARY CATALOGUE CARDS which TBT students had not studied, but did not test the identification of abductor pollicis longus and extensor pollicis brevis which they had; the CAT tested DICTIONARY PAGE which TBT students had not studied, but did not test the playing of major, minor, and diatonic scales, which they had; the CAT tested BEGINNING WORDS/TITLES which TBT students had not studied, but did not test the drawing of cumulative records distinguishing fixed interval reinforcement from variable ratio reinforcement which they had.  Given the CAT's testing of some insubstantial topics that the conventional curriculum teaches, and its failure to test some of the substantial topics that TwelveByTwelve taught, Kirsten's and Alexa's scores might be considered even more remarkable than they might have initially seemed.

The Last Word

The conclusion to which the TwelveByTwelve Pilot Study points is that the vast majority of children, including ones who in the conventional classroom are considered recalcitrant or unpromising, can accomplish all the skills described on the instant TwelveByTwelve Pilot Study page, and a great many more, and while experiencing a reduction in strain as compared to what they had been experiencing in conventional schools, as well as a boost in enthusiasm.  The American standard of No Child Left Behind is among the many educational goals that as yet have only been set, but which TwelveByTwelve would be able to realize.

TwelveByTwelve (TBT)      Pilot Study      Explicit Transformational Syllabus (XTS)     Fundamentals of TwelveByTwelve