Aesthetic Benchmarking

I’m curious if anyone has any experience with aesthetic benchmarking, and how you developed your criteria? I’m trying to developing criteria to measure/compare aesthetics in a quantitative and/or qualitative way. The goal is to measure an existing product on a variety of different criteria and then do a Gap Analysis (a comparison of criteria to see if a product currently exceeds, is equal to, or needs improving when compared to what might be considered it’s competitive best in class for a specific line of the criteria).

As an example let’s say I’m trying to benchmark automotive interiors in the luxury sedan class. So I’m looking at the Audi A4, BMW 3 Series, Lexus IS, Cadillac ATS, and Acura TSX; what types of specific criteria might I use? Maybe I should be asking what specific criteria does each design most differ? What areas do they separate themselves most in?

I would expect the criteria would look at the following areas (feel free to add more or refine):

  • Trends
  • Design language
  • Color, materials, quality of construction
  • Target market
  • Function (this one is easy and why I’m specifically trying to find some measure for aesthetics)

    Any guidance, insight, experiences, or other sources of information would be much appreciated. Questions are welcome.



Tucker, I think you may need to break down what you are trying to find into smaller chunks.

Also, know the any kind of surveying would be best done in person. The products will have to be experienced, touch, felt, smelt, for such a subjective analysis to have any kind of dimension.

If I were doing something like the example you have above, I’d look to vary the sample set more: i.e. 3 American Cars, 3 German, 2 Japanese. Then take all the logos off (hard with cars because so many people know them) and carefully script questions that get at the meat of what you are trying to learn about. For example: What country does this car come from? It would be interesting to know if people got that right, and from this you may infer if the brand is doings its job with its design language (if its goal is to look like the country it comes from).

It is difficult for people to comment on something as broad as aesthetics. Even the terms luxury or premium are too loaded and broad. I would craft questions around what looks/feels precise? What looks/feels crafted? What looks/feel American, Japanese, German? How old is the person who buys this in your opinion? What activities is the person who buys this vehicle into?

If you can ask these types of things of enough people, and keep the questions as specific and closed as possible, you can start to have a body of data to synthesize into insights. The key is always in the synthesis.

Yo, thanks for the response. What you are proposing sounds great and would be beneficial to the understanding of the example I posed. I think I need to clarify my question in terms of the big picture intent though.

  • I’m looking at whether a product has become aesthetically/functionally dated
  • It was determined that benchmarking (out of my control) would be a good way to go about looking at similar industries and comparing the product to those that have more recently been redesigned/updated (this is where the Gap Analysis I mentioned above comes in).
  • It needs to be noted that I’m unable to interview/observe the end user using the product (another reason we are doing benchmarking of the industry and not just user focused ethnographic research)

With that being said I have a few additional questions I hope will continue this conversation.

What criteria might you use when looking at if a product needs a redesign/refresh?
For example in the automotive industry they typically have a 4 year refresh cycle, how do they justify this (what if sales data wasn’t a driving factor)?

Trying to quantify/qualify if something is outdated…

Regardless of the criteria, would you be the one who decides if it is out of date or not?

The Gap anaylysis/research should prove that it is out of date.

I think as designer we know that when a successful product has been on the market for roughly 5 to 10 years (in it’s current form), that we a probably missing out on new sales and opportunities for improvement. A redesign/update would allow us to make a more relevant product for todays market and in turn draw existing and new customers to re/consider the product. The new marketing buzz could then lead to an increase in sales (when properly designed for the target market/customers).

I’m looking for sources or experiences that others have used when trying to justify redesigns like this.

This is a key starting point as Trends, Design language, Color, materials, quality of construction, Target market, can be too broad.
One recommendation is to audit multiple products along semantic differential scales. For example:
Economical -5 -4 -3 -2 -1 0 +1 +2 +3 +4 +5 Luxury
Low Quality -5 -4 -3 -2 -1 0 +1 +2 +3 +4 +5 High Quality
Serious -5 -4 -3 -2 -1 0 +1 +2 +3 +4 +5 Fun
Modern -5 -4 -3 -2 -1 0 +1 +2 +3 +4 +5 Old
Contemporary -5 -4 -3 -2 -1 0 +1 +2 +3 +4 +5 Dated (or is it Classic?)

It would help to establish “Best-in Class” attributes for area of focus. Plotting ‘Best in Class’ as zero would readily identify who is over, and underachieving. As FYI automotive brands per your example are often measured against key personality attributes (spirited, imaginative, daring…).
There are so many factors that drive product redesign/refresh. Current product: no longer meets customer wants and needs, becomes too costly to produce or deliver to market, is no longer price competitive, is not profitable, no longer meets quality expectations, etc. Consumer, market, or technology shifts can also leave a company or product behind.
Back to your automotive example, a 4-year refresh cycle is very much determined by effort it takes to take a concept through development, manufacturing and to market. It requires significant investment of resources, especially if company has many platforms/brands in its portfolio. For what its worth, most auto brands typically plan for yearly updates, and one major refresh during platform lifecycle. I’m not sure how you would capture this, but I hope some of this helps.