“Say Cheese!”: Using Face Averaging to Track the Rise of the Photo Smile

The centerpiece of this post is an animation showing the rise of the “say cheese” smile in American portrait photography.  In three seconds, it displays the change in typical facial expression over the first four decades of the twentieth century as condensed from nearly fifteen hundred actual photographic portraits.  If you’re the sort of reader who needs to flip to the back of a murder mystery first thing to find out whether the butler did it, feel free to scroll down right now and take a look.  But for those who value a narrative arc, I’d like to give you some background to explain how I created the animation and why I think it’s interesting.

---

I.

Let’s start with a couple snippets of anecdotal evidence.  Last year I bought three photographs at a Louisville antique store, ranging in date from the 1860s through the 1880s.  They had caught my eye for various reasons—I’ll let you guess about what they were—but what impressed the lady at the cash register about them was that none of the subjects in them was smiling.  “They didn’t have anything to smile about,” she remarked as she rang them up.  “No decent health care, no indoor plumbing.”

She was being funny, but I think there was a kernel of sincerity in her banter.  For the average twenty-first-century viewer, the facial expressions of people in nineteenth-century photographs can indeed come off as rather somber.  It might be tempting to infer from this that everyone back then really was unhappy and had good reason to be unhappy.  On the other hand, if people of the nineteenth century could somehow have passed judgment on our own twenty-first-century photographic portraits, I suspect they’d have drawn some equally stark conclusions about us from them.  Of course, they were never actually in a position to do that; for better or worse, we can’t send our pictures back in time.  But here’s a telling remark published just a few years after the fin de siècle:

I take up an illustrated paper in which lovely women show their teeth on every page. No doubt they were asked by the photographer to “look pleasant”; they have done their best, poor things, but if their great-grandchildren should ever see these pleasant smiles, they will say with one accord, “Our great-grandmothers must have been a lot of idiots to grin in this manner.”^[1]

If the critic who wrote these lines could have seen a sample of photographic portraits from 2014, I imagine he and his contemporaries would have deduced that we must be a lot of idiots to grin for the camera as we do.  We’re plainly dealing with a mismatch between two cultural norms, ours and theirs: the seeds of a miscommunication across the centuries.  To us, they look unhappy, or at least drearily serious.  To them, we would apparently have looked like dimwits.  Maybe some future generation will agree: “Look at the silly smirks on those morons who squandered all that oil and gave us global warming.  You can tell just by looking at them that they didn’t have a care in the world.”  But I digress.

The cultural change at issue here has attracted occasional scholarly attention.  Back in 2005, Christina Kotchemidova stated in a piece published in Critical Studies in Media Communication: “The ‘Say cheese’ smile characteristic of contemporary popular photo culture has a history. Although its exact starting date cannot be established, we can easily contrast it to the invariably serious facial expression of nineteenth-century photos.”^[2]  I agree that it’s easy to notice the contrast Kotchemidova has in mind here; anyone who spends much time around historical photographs has surely picked up on it.  But I don’t know that there’s been any obvious way for anyone to document this contrast.  Sure, we can tell informally from looking at lots of examples that one cultural norm gave way to another, but the transition was presumably somewhat gradual, and wouldn’t have had an “exact starting date”—some precise moment when photographers and their subjects all unanimously decided to change the way they handled facial expressions.  Moreover, the “say cheese” smile isn’t a simple yes-or-no thing but varies in intensity in ways that would be difficult to quantify, and its rise to normalcy would have involved large-scale tendencies rather than invariable rules: people did smile for earlier photographic portraits every now and then, and not everyone smiles for them today.  We might venture the grinning gentleman with the mustache in this circa 1890s tintype and Nicolas Cage’s gloomy high school yearbook photo as historical outliers:

With all those complications in mind, how could we ever go about methodically and accurately tracking the rise of the “say cheese” smile?

In this blog post, I’d like to explore one promising strategy: using face-averaging software to generate averages from groups of photographic portraits taken during successive periods in time as a means of documenting and displaying long-term changes in facial expression.  (Previously on Griffonage-Dot-Com: I’ve written about face averaging as a historical technique, and about its application to art history, and I’ve proposed the name prosopochrony for the combination of face averaging with a time base.  As before, the software I’ll be using to create my averages is the deluxe version of Abrosoft’s FaceMixer.)

To get a sense for how face averaging can be brought to bear on the issue of the “say cheese” smile, contrast these averages of the portraits of thirty-nine young women and 107 young men from the Liljenquist Family Collection of Civil War Photographs at the Library of Congress—

—with these averages of the senior yearbook photos of the Kewanee High School Class of 1995:

Notice any difference in attitude?  Yeah, I thought so.

True, some Civil War era subjects had jauntier expressions, and some high school students in 1995 glowered like Nicolas Cage, but the expressions seen here are averages—the expressions most typical of the data sets, and of the moments in time those data sets represent.  We can say, then, that the first pair of images shows a fishlike stare typical of American portraits taken around 1861-1865, and that the second pair of images shows a grin typical of Midwestern high school yearbook photos in 1995; and we could generate similar averages from comparable data sets for any year in between, reducing large and unwieldy masses of specimens into single, easily juxtaposable images.

Yearbook photographs are a wonderfully convenient source of material for this kind of experiment, so let’s pursue things a little further with the Kewanee High School yearbooks (which I chose purely because a long run of them happens to be available online).  Since the female average for 1995 shows a bigger, toothier smile than the male average—and hence presents a starker contrast with the nineteenth-century grimace—I’ve opted to focus my experiments on senior-class women.  Here are averages I created for the twelve years from 1989 to 2000 (basically the nineties, with an extra year thrown in on either side for good measure):

We can also view the averages in sequence as an animation:

There’s a gradual change in appearance over the twelve-year period—the averages for 1997-2000 resemble each other more than they do any of the averages for 1989-1996—but I see no significant change in expression; any difference appears limited mostly to makeup, hairstyle, and perhaps ethnicity.  The “say cheese” smile is consistent throughout, which is what I’d have expected: as far as we know, this cultural norm was well-established at the start of the nineties and hadn’t varied by the end of the decade.  Where there’s no change, we see no change—that is, apart from minor shifts in angle and a slight twitchiness about the mouth.  That’s an important control for our experiment, a baseline for what to expect when the parameter we’re interested in is holding steady.

But what happens if we compress our time scale and select a span of years that—unlike the nineties—overlaps the rise of the “say cheese” smile?

---

II.

Let’s find out.  There’s a pretty complete run of Kewanee High School yearbooks online from 1904 (pre-smile) through 2002 (post-smile), but it’s missing the years 1918, 1919, and 1920—publication of the yearbook had probably been suspended during the First World War and took a while to get going again afterwards.  To fill in those missing years, and to broaden my source base as an added bonus, I turned to Kokomo High School and Sheboygan High School (which eventually split into two schools; I went with Sheboygan North after the split).  Then I set to work creating averages for years chosen at five-year increments, with two schools available for 1904, 1909, 1914, and 1919, and all three schools available for 1924, 1929, 1934, and 1939.  Here are the averages for each school displayed separately (a black square means a yearbook wasn’t available for that school for that year):

In each case, I included all published pictures of senior-class women except for the few full profiles (the software I’m using can’t handle them)—there was no arbitrary selection on my part.  There’s not much change visible between 1904 and 1924, although towards the end of that span we start to see hints of a tepid smile.  In 1929, Kokomo takes the lead with a nice big toothy smile, while Sheboygan remains tepid and Kewanee is still dead serious.  We can read this as evidence that conventions were in flux at this point; either the high school cultures were somewhat different, or the prompting of individual photographers was enough to nudge subjects in one or another direction.  In 1934, each average is unambiguously smiling, and by roughly the same modest amount: a consensus seems to have been achieved.  By 1939, each average is unambiguously baring teeth; Kewanee has a full-fledged “say cheese” smile, and Kokomo and Sheboygan aren’t far behind.  That’s where I am now.  I haven’t yet started work on 1944, but based on the results so far, I’m wondering whether it might not be more illuminating to go back and fill in each individual year from 1920 to 1938.  I’m open to both suggestions and help.

It’s clear that there are differences between schools that are worth trying to document and understand.  However, I also created averages of all three schools combined.  I gave equal weighting to individual portraits rather than to schools, so larger classes are more heavily weighted than smaller classes.  Also, the later images are based on larger data sets than the earlier ones, partly because the earlier images are based on two schools each rather than three, but mostly because of an increase in high school class sizes.  Specifically, I averaged 45 total portraits for 1904, 62 for 1909, 57 for 1914, 106 for 1919, 168 for 1924, 256 for 1934, and 413 for 1939.  (My past experiments have suggested that fifty portraits should be enough to produce a respectable average, and that greater numbers tend to have only a smoothing effect—compare the averages made from 25, 50, and 100 baseball card pictures in my earlier blog post, “Face Averaging as a Historical Technique.”)

And finally, the moment you’ve all been waiting for: here’s an animation showing the change in average facial expression over the whole period, as promised at the beginning of this blog post:

Remember: this isn’t a simulation.  It’s a cumulative display of actual photographic data.  To produce it, I organized all the averages shown above by date in the order Kewanee, Sheboygan, Kokomo, and three-school composite, and then re-averaged these in successive sets of four, as follows (with one single-school average shown in boldface to highlight the pattern):

• 1904 Kewanee + 1904 Sheboygan + 1904 Composite + 1909 Kewanee
• 1904 Sheboygan + 1904 Composite + 1909 Kewanee + 1909 Kokomo
• 1904 Composite + 1909 Kewanee + 1909 Kokomo + 1909 Composite
• 1909 Kewanee + 1909 Kokomo + 1909 Composite + 1914 Kewanee, etc.

That process gave me twenty-five averages; for the final animation I interposed two tweened frames between each average and then neatened up two slightly “jerky” segments.  The result still has a little unsteadiness comparable to that of a clumsily handheld video camera, but I’d say it’s not bad for a forty-year time-lapse image—and with work I’m sure I could improve on it.  We appear to see two things happen:

1. The subject’s eyes are initially averted but then turn towards the viewer.
2. The subject smiles, culminating in a “say cheese” smile—or something close to it.

---

III.

Of course, “say cheese” is just a phrase photographers conventionally use to get subjects to form a broad smile with teeth exposed—the phrase itself isn’t really the point.  That said, it’s often claimed that this practice supplanted an earlier one of asking subjects to “say prunes.”  There are a few primary sources around to back those claims up.^[3]  However, during the late nineteenth and early twentieth centuries, the prevailing convention seems rather to have been for photographers to ask their subjects to “look pleasant.”  What exactly that meant was open to interpretation, but in 1890, a joke appeared in print headed “Force of habit”: “Amateur Photographer (before the Sphinx): ‘Look pleasant, please, and don’t smile.'”^[4]  And that’s by no means the only textual evidence that smiling was not equated with looking pleasant in a photograph in the nineteenth century.  Mark Twain wrote: “A photograph is a most important document, and there is nothing more damning to go down to posterity than a silly, foolish smile caught and fixed forever.”^[5]  There’s also this passage from a British book of 1876 in which a man disparages the “ugly mugs” displayed in a woman’s photograph album:

“How she must despise her friends as she looks over them; from that girl with a leer, who is holding up a pocket-handkerchief with this motto on it—

‘Yes, this is my album,
But learn ere you look,
That all are expected
To add to my book.’

“Pah! I can’t repeat the rubbish; the two last lines say something about ‘leaving your portrait for others to quiz.’”^[6]

I can’t be sure what specific image of a “girl with a leer” the character is supposed to be knocking here, but he’s definitely describing a particular type of CDV that was designed to go in the first page of an album with a rhymed appeal for anyone looking at it to give the owner a photograph of himself or herself in return for the chance to look at other people’s photographs; these weren’t used much if at all in the United States, but they seem to have been common in Great Britain.  The most frequently encountered British version I’ve seen with one girl, as opposed to two, is shown below in two slightly different variants:

Granted, other less common versions exist, at least one of which shows a woman with a toothier smile (yet another has the mouth hidden entirely behind the frame).  But the quoted passage still hints that the smile seen above—understated and demure by twenty-first-century standards—would have been regarded in 1870s Great Britain as an over-the-top “leer.”  It’s certainly a more pronounced smile than we see in 99.9% of the photographic portraits taken during that period.  The subject probably smiled as much as she did only because she was posing not for a photographic portrait as such, but for a piece of “album filler.”  Whatever was going on here, it had little to do with the model’s self-presentation as a complex human being.

And I can point to yet more evidence drawn from photographs themselves that people in the nineteenth-century avoided smiling when posing for the camera.  Women who took children to be photographed often ended up maintaining physical contact with them—holding them on their laps, touching them with an outstretched hand, and so forth—to keep them complacent and still during the shoot so that they wouldn’t end up as blurry messes.  These women usually tried to remain inconspicuous and out of sight, and the results are known among enthusiasts today as “hidden mother” photographs.  Sometimes the woman’s face was covered with a blanket, and sometimes the photographer has scraped it away or otherwise effaced it in postproduction; but often it’s just cropped mostly out of the frame, as seen in the two circa 1870 examples below (the CDV on the left was taken by G. F. Maitland of St. Catharines, Ontario, and the tintype on the right is unsigned):

The “hidden mothers” in these two photographs have the broadest, toothiest smiles I’ve seen anywhere in the studio portraiture of the period.

But what distinguishes them from the people seen in other studio portraits is that they didn’t think they were being photographed.  Their default inclination when sitting with their children in a photographer’s studio was apparently to smile: maybe it was a happy, fun occasion.  But when these same women—or others like them—thought of themselves as subjects of photographic portraits, they must have consciously suppressed that urge to give us images that look so consistently like these:

I’ve seen four explanations offered for the transition from the somber expressions in nineteenth-century photos to the “say cheese” smiles of our own time:

• Improvement in teeth.  Most people would have been too self-conscious to expose their bad teeth before the era of modern dental care and orthodontics.
• Reduction of sitting times.  It wouldn’t have been possible to hold a natural smile for the length of time required to take the first Daguerreotypes, much less the time needed for someone to paint a portrait.  Unsmiling portraits thus became the norm.  With reduced photographic exposure times, it finally became possible to capture natural smiles.
• Change in connotations.  As Nicholas Jeeves writes: “By the 17th century in Europe it was a well-established fact that the only people who smiled broadly, in life and in art, were the poor, the lewd, the drunk, the innocent, and the entertainment.”^[7]  Only when photographs of broadly smiling celebrities became commonplace did people begin to emulate them in posing for the camera.
• Kodak advertising.  Christina Kotchemidova writes: “[I]nstead of viewing the advent of the smile as a process of ‘mouth liberalization,’ which presupposes a natural desire to smile for a picture (although one long culturally banned), I propose looking at this twentieth-century norm as a cultural construction.”  She attributes the change largely to commercial efforts to promote informal snapshot photography by associating it with fun and happiness.^[8]  By contrast, the professional studio setting of early portrait photography had given it an air of seriousness and formality.  I suppose the experience of smiling for snapshots could eventually have spread to smiling in other photographic contexts.

We can now complement these hypotheses with some preliminary observations about timing based on the face-averaging experiments:

• There was a gradual, barely perceptible shift towards a weak smile over the first quarter of the twentieth century.
• A toothy smile crops up in 1929, but it’s the norm in only one of three schools studied.  This kind of smile had apparently become acceptable for yearbook portraits at this point, but there was no clear majority “saying cheese” yet, even if we begin to see an effect on the cumulative three-school average.
• By 1934 the smile had spread to all three schools, at a consistent but modest level in each.  If we’re looking for the point when it became more common to smile for the camera than not to smile, that seems to have happened sometime between 1929 and 1934, contemporaneous with the onset and depth of the Great Depression.
• By 1939 the smiles had become bigger and toothier everywhere, implying that the tendency to smile for the camera had intensified during the second half of the decade.

Bottom line: the “say cheese” smile became a prevailing cultural norm right in the midst of the Great Depression—a time when people arguably had less cause for happiness than at any other point in recent history.

And that might make a certain degree of sense.  The people of that era could have felt a greater need to seem happy—to put on a cheerful front—precisely so they could avoid exposing a distressingly grim emotional state to themselves and to others.  Such deception had been associated with smiling for the camera before.  Back in 1907, a critic had written of women photographed with toothy smiles for illustrated papers:

Why is this attempt at looking pleasant so distasteful to us? Because we see too often that it is only an attempt, and beneath the smile we find weariness and other expressions which speak of anything but pleasure. It was Darwin who first pointed out to us how to tell whether a smile was a genuine one or not. But it would be unkind to repeat his discovery, for are there not thousands of people who have to make their living by smiling? and after all there is something heroic in looking pleasant when the wolf is tearing at one’s heart.^[9]

“Something heroic,” indeed: during the First World War, we find the act of smiling for the camera framed as a brave show of holding up under adversity.  Witness these captions:

• “A Group of Women Who are Employed in Another British Brick Yard. In Spite of Their Laborious Task and the Hardships Which They are Suffering Because of the War, They Find Occasion to Smile for the Camera Man.”^[10]
• “One hundred and third Infantry, wounded but still able to smile for the camera. Near Mery, France. July 22, 1918.”^[11]

In the same spirit, the Great Depression would have been a prime opportunity for “looking pleasant when the wolf is tearing at one’s heart” on a grand scale.  Or there may have been other cultural trajectories in play: a higher value placed on casual informality, say, or a new feeling that if you didn’t project a happy persona, there was something wrong with you.  Increasingly widespread exposure to snapshot photography might have played a role too.  Whatever the explanation may be, I think the results of my ongoing face-averaging experiments can at least help us pinpoint what we need to explain.  And there’s plenty more to investigate in this way: differences in gender, geographic region, age, and genre all come to mind.

---

References

1.  Frank M. Sutcliffe, “On Looking Pleasant,” The Amateur Photographer, Nov. 5, 1907, p. 439, online here.

2.  Christina Kotchemidova, “Why we say ‘cheese’: Producing the smile in snapshot photography,” Critical Studies in Media Communication 22:1 (Winter 2005), 2-25, quotation at 2.

3.  One article from 1846 actually advises photographic subjects against “making up a face for the occasion” but grudgingly adds: “If the ladies…must study for a bit of effect, we will give them a recipe for a pretty expression of mouth—let them place it as if they were going to say prunes.”  See Andrew Winter, “The Pencil of Nature,” People’s Journal 2 (1846-47), 288, online here.  An account from 1908 states: “Thursday the boys got their picture taken and were told to say prunes at the moment the camera was snapped, so as to have pleasant expressions. This accounts for the grin on Elder’s and Newcomb’s faces.”  See the Numismatist 21 (1908), p. 347, online here.  Finally, we read that “many old-time operators were in the habit of asking their sitters to say “prunes” or “prism” before making an exposure”; that’s in an article of 1935 in the British Journal of Photography that I’ve seen only in Google Books snippet view.

4. Photographic Times, June 20, 1890, p. 302, online here.

5. Elizabeth Wallace, Mark Twain and the Happy Island (Chicago: A. C. McClurg & Co., 1913), p. 34, online here.

6. James Hain Friswell, Other People’s Windows (London: S. Low, Marston, Seales & Rivington, 1876), 304, online here.

7. Nicholas Jeeves, “The Serious and the Smirk: The Smile in Portraiture,” Public Domain Review, online here.

8. Christina Kotchemidova, “Why we say ‘cheese’: Producing the smile in snapshot photography,” Critical Studies in Media Communication 22:1 (Winter 2005), 2-25, quotation at 3.

9. Frank M. Sutcliffe, “On Looking Pleasant,” The Amateur Photographer, Nov. 5, 1907, p. 439, online here.

10. Brick and Clay Record, October 23, 1917, p. 759, online here.

11. Catalogue of Official A. E. F. Photographs, Volume 2,  online here.