Tuesday, August 19, 2014

And Then What Happens? Measuring Arts Impact

"However beautiful the strategy, you should occasionally look at the results." - attributed to Winston Churchill

If we want broad support for the arts, we have to be able to answer the question everyone asks, or wants to ask - even if they don’t realize it: “And then what happens?”

Art as entertainment means selling tickets and memberships, or getting funding from donors who care a whole lot about your kind of art.

But, if we want everyone else to care about the music, museums, murals, performers, dance, theatre, and so on -- then we have to make it relevant to them, whether or not they are (or think they are) goers.

And that’s especially true if we want art to be considered a public good - worthy of revenue forgone (a tax credit or deduction) or taxation and fees dedicated to the arts.

How do we do that?

Our research finds that most people believe the arts create surprising ripple effects of benefits: creating vibrant places we all want to live, visit, work, and invest - places where people meet and get to know one another, in ways that other events can’t connect them.

And it turns out that people can see that if the neighborhoods in their community benefit in this way - it’s good for everyone who lives there, even people who don’t go to the arts themselves.


We need to start talking about the arts in ways that bring these benefits to life. Because while people believe the arts create these benefits, it’s not their natural way (their first way) of thinking about the arts. If we want these benefits to be front of mind, we have to frame our communication about the arts to make it so.

And it’s important that it really IS so.

Most of us who work in the arts, if we can get past the ‘art for art’s sake’ view of our world, already believe that the arts create vibrant places and bring people together. We see it happen all the time. Right? But, many personal experiences do not equal fundable data.

Now, how do we measure these social impacts of the arts?

In the past few weeks, I’ve had three experiences with social impacts of free arts events and observed a fourth that seems especially relevant.

Stories of Route 66 -- ID LIVE! Festival

Co-created painted chairs offered as prizes at ID LIVE!

In mid-July, I went to Albuquerque to provide some hands-on-help and observe a special festival event presented by a large collaborative arts initiative, Stories of Route 66.

Stories of Route 66 is located in a very diverse neighborhood of ABQ, the International District. It's had other -- more negative -- names (i.e., The War Zone), and Stories of Route 66 is designed to create change through the arts, to shift the way people view the neighborhood. The ID LIVE! festival celebrated some initiatives of the various partners with three days of events:

  • unveiling of corner gardens with permanent and temporary arts installations like sidewalk painting, transforming corners that are usually empty lots,
  • pop-up parties in empty lots and on closed streets, including one with a pop-up party toolkit in a truck created by the University of New Mexico students, and
  • performance arts and film created by a diverse group of residents who met for six months, making art together every week.

All of these events came together on a very limited budget and with the commitment of a strong group of organizers and volunteers. It was quite an impressive effort by a relatively small group of people who put in a lot of time. Still, this made dedicating energy and human resources to measuring outcomes challenging. (This is familiar, right?)

Pop Up Street Party

We wanted to know whether the special weekend of arts installations in various locations throughout the neighborhood would have any impact on how the residents and visitors perceive the neighborhood and their willingness or capacity to invest in a longer-term, future effort to create permanent art installations.

The team had already decided to use a passport to encourage people to come to the festival and to provide a sort of guide featuring some of the many events. This was especially important because the area is very large and there were great distances between event sites.

The team also planned to use the passport to gather information from attendees - providing  a way to contact them in the future for efforts to place art permanently, and to gauge any changes in perception about the neighborhood.

ID LIVE! Passport

Volunteers distributed some passports as invitational doorknockers in the neighborhood. At each of the passport activities volunteers were asked to offer passports, punch them for the current activity, and encourage people to attend other events.
As an incentive, passports could be turned in at the end of the festival and prizes would be randomly awarded to passport holders who attended at least three events. And if they completed an additional, short survey at that time - passport holders would be eligible for a grand prize of $100 cash.

We spent some time debating how many punches a passport holder had to have to be eligible for a prize. Since we really wanted the data (feedback about perceptions and contact information), we probably set the bar too high by requiring three hole punches.

Stamp Station at the Pop Up Street Party

It was an especially hot weekend and there’s very little shade in the neighborhood, so only the most committed people went to more than one event. The organizers created a terrific map, but it still required time and effort for people to visit more than one location, since no one could see the site of another event from any one location.

Chalk Art Created by Community

In the end, we got limited feedback. Since we didn’t have observers at every site, it’s hard to know exactly what the barriers to returning the passport might have been. The events were understaffed for focusing on the passport and it was no one’s top priority to be sure that attendees got a passport or understood how it could be used. And as noted above, the bar was very high for our goal of gathering information. Finally, passports were a sort of souvenir and some attendees seemed to really like them, objecting to the hole punches which obliterated the images related to the events. Perhaps they didn't return the passports because they wanted to save them. (That would be a good indicator - if we had a way to measure it!)
This experience gives us good information for, and more interest in, a follow-up evaluation design.

Museum Camp

Museum Campers on the steps of the MAH

Immediately after ID LIVE! in ABQ, I traveled to California for Museum Camp -- produced by the Santa Cruz Museum of Art and History with Fractured Atlas (aka the incomparable Nina Simon and Ian David Moss). The following description of the planned event is cribbed and edited from the museum website.

The 2014 Museum Camp at the Santa Cruz Museum of Art and History theme was social impact assessment. The goal was to develop creative ways to evaluate work designed to build and transform our communities. It brought together teams of diverse people from across many disciplines in shared learning and doing around research and social impact. The focus was on social impact in communities, and the organizers encouraged teams to look at complex outcomes–like safety, cohesion, compassion, and identity–that are not commonly covered in our standard evaluative practices.

Participants were assigned to teams of four (by the event organizers to ensure team diversity) and we selected social outcomes of interest that the organizers suggested we would like to measure. Each team had about 48 hours to determine an outcome of interest, identify an indicator of that outcome, develop a creative evaluation tool to measure it, perform the measurement, and analyze the results.

There were 100 creative people from diverse arts, social justice, and community development organizations who wanted to experiment and push their practice through an active, collaborative professional development experience. Registration was based on a competitive application process.

Every team was assigned a location or event to conduct their research. The whole camp played a white-elephant style game for the assignments. My team ended up with a Friday Night Free Beach concert. We struggled mightily to come up with a research question that we could all be passionate about, that also fit the location.

Our location was described this way: “Boardwalk - free Friday night concerts at 6:30 and 8:30, featuring heavy metal band Y&T” and we were told that it would fit well with a hypothesis about “live performance, free events, belonging”. A little research revealed that Y&T is a hair band that had some hits in the eighties.

Team Hair Band (High Five for Santa Cruz)

Nick Stillman-Arts Council of New Orleans
Alice Briggs-Ceredigion Museum
Diana Kapsner-Santa Cruz MAH
We were interested in whether a free concert increases a feeling of affinity, pride, or belonging to a place - and in particular, whether people who happen-upon the concert feel identity more strongly than people who come with intention. Does serendipitous art have a special social impact?

Our method for assessing was to invite people with a questioning “high five for Santa Cruz?” and we developed indicators to suggest people who came with intention, happened-upon the concert, or were completely unaware of the concert.

High Five for Santa Cruz?
We were tripped up by the private boardwalk security who didn’t want us on the boardwalk. So, dressed in our huge sandwich boards and divided into two teams of two people each, we took up spots on the beach in the midst of the concert footprint and on the other side of the arcade where visitors could not see or hear the concert.

We quickly realized that the spot on the beach was almost exclusively populated by people coming with intention. (There were WAY more people wearing Y&T t-shirts than we anticipated!)

And we realized two other things:

  • People LOVE the free concert series and they were REALLY happy to be there.
  • There’s a social expectation related to “high-fiving” ~ nearly everyone responded to our prompt to high five for Santa Cruz.

So we made an adjustment about a half hour into the experiment and started noting enthusiastic high fives as a separate category of response.

And for the last half hour, the beach team scouted the edges of the beach seating area for people who might happen upon the concert, since those descending the steps from the boardwalk were nearly all coming with intention.
Searching for Serendipity
Our findings? Looking at the numbers, our hypothesis was proven by the results. But the number of happen-upon responses was so small it’s hard to rely on the results. And we determined that it was difficult to pull apart the reasons participants would respond positively to our request to high-five for Santa Cruz.

Anecdotally, we noted the numerous people who verbally expressed a deep love for Santa Cruz and a feeling of pride in and identity with the city. Even non-residents and former residents took time (without prompting) to explain their connection to the place and their love for this long-standing concert series.

Museum Camp made me want to play ~~ that is, experiment ~~ much more with measuring social impact of the arts. Just get out there and try stuff.


My third experience is not quantitative at all. We weren't trying to measure the impact of the art at this event.

The morning after my late night return to Cincinnati (from ID-Live and Museum Camp), my non-profit arts group, Art on the Streets, produced its third ArtWalk  - a co-created mural in a crosswalk.

For ArtWalks, we seek community input as inspiration for artists who create the designs -- and then we chalk the crosswalk design and invite the public to paint the street.

The ArtWalk at Orchard and Main was painted at a community festival called Second Sunday on Main. Most people who painted found the art-making serendipitously.

Painting the Orchard Street ArtWalk

One father, watching his two children paint, said to them, “Do you know how lucky you are? Cincinnati is so amazing! You are so lucky to grow up here.”

As I told him, this is exactly the impact we hope to discover from art in public places. We dream, we fundraise, we create - we present the art or we make it together. And then what happens?

We’ll just have to keep trying to figure out how to measure our success.



LumenoCity photo by cincinnatirefined
The fourth event is one I observed purely through the lens of social media. LumenoCity happened in my home town of Cincinnati, but I was at Museum Camp at the time. It's a free illuminated concert, produced and performed by the Cincinnati Symphony Orchestra in an outdoor venue with the light show playing on our iconic and historic Music Hall.

The first year of LumenoCity in 2013 proved wildly popular and generated world-wide recognition for an engagement event produced by a classical music organization when 35,000 people attended two nights of concerts. The event transformed the neighborhood and brought thousands of people into an area many hadn't visited in decades, a place still full of empty and abandoned buildings, still considered dangerous by many area residents. Just after the 2013 event, there was a sort of LumenoCity effect, with real estate prospectors trolling the area with clipboards.
photo via cincinnati.com
In 2014, the producers decided to ticket the event - even though admission was still free. They explained that this was being done for safety reasons; they had concerns the previous year when so many people crowded into a small area. 

There were some major hiccups with the ticketing, and the media stories (across all channels) tended to reinforce existing default perceptions of the symphony as elitist. Many people who wanted to attend could not get tickets inside the gated public park venue, while the symphony season ticket holders and donors were given access to tickets in advance.

Nevertheless, on the weekend of the event the social media reaction in my stream (again, on multiple channels) was extremely interesting. 

 Many of the posts expressed love for and pride in Cincinnati, suggesting that, despite the difficult ticketing issues - and possible negative impact on perception of the arts for some people, those who attended, or saw photos and videos, were left with a new and strong sense of pride and affinity for place.

Now, if we could just figure out how to measure WHETHER THIS HAPPENED!

See what I mean? A simple search for #lumenocity Cincinnati produces this, and more.

No comments:

Post a Comment