's multicar comparison results aren't magic. Our expert judges spend a week driving and comparing vehicles in the best way anyone can evaluate competitors: back to back to back. Judges include editors and, often, an in-market shopper who take each car on identical routes to uncover the little idiosyncrasies that in some cases can determine the difference between a winner and a loser.
Our judges' opinions account for a large portion of determining a contestant's score, but depending on the class, we also include objective scoring in the form of fuel economy, car-seat fitment, safety features and/or crash worthiness, acceleration and braking. Below is how we scored SUVs in the 2018 Luxury Compact SUV Challenge.
Subjective Scoring Categories
Interior Quality
Interior quality considerations include whether it seems appropriately luxurious, rating the appearance and feel of the surfaces and overall craftsmanship of quality — judges note if the materials are rich and authentic or a cheap imitation. Another consideration is whether the quality drops from the front to rear seats.
Front-Seat Comfort
Legroom, headroom and knee room are all considered when determining front-seat comfort. So is whether the cushioning is comfortable and if the seat can be adjusted appropriately. Features we look for include heated and ventilated seats, the extent of passenger seat controls, and memory and massaging functions.
Backseat Comfort
Along with dimensions, backseat comfort is determined by cushioning, support and whether there's a large center floor hump that could crowd foot room, as well as whether it's easy to get in and out of. Backseat feature considerations include whether the seats slide or recline and if it has adjustable air vents, rear climate controls, and heated or ventilated rear seats.
Cargo Storage
We've reported how cargo specs can lie, and cargo storage is much more than a square footage number. When judging cargo room, we're looking for how usable the space is and how easy it is to load and retrieve objects, as well as features including whether the backseat can fold to extend the cargo space, if there's a power liftgate, cargo-area releases for the folding backseat and if the folded backseat creates a flat cargo area.
In-Cabin Storage
As the size of our smartphones and other mobile devices grow, it's important to have the space to accommodate them. We evaluate those and other storage options in the cabin, including open and covered storage and if there are enough cupholders or a sunglasses holder.
Multimedia Features
Judges consider ease of use, screen quality, connectivity, responsiveness and controls. In addition, we note the presence or absence of Apple CarPlay, Android Auto, Amazon Alexa, onboard navigation systems, Wi-Fi, wireless smartphone charging, mechanical volume and tuning knobs, head-up displays and virtual gauges. We measure screen sizes and record the number of USB ports and 120-volt household-style outlets.
Convenience Features
The convenience features category accounts for what's provided in our specific test vehicles, such as keyless access, push-button start, remote start, one-touch windows, moonroofs, heated steering wheels and more. Also a consideration is how well they work: Is the moonroof bigger or smaller compared to competitors? Are all windows one-touch up or down, and does the entire heated steering wheel warm up, or just a portion?
Handling
Judges drive the car on the same route to determine how well each car corners, determining if it rolls (i.e., leans), whether it feels planted and confident or uneasy, requiring too much effort to drive cleanly through a corner. Steering and all-wheel-drive execution also play a part.
Powertrain
Rather than a reflection of the vehicle's quickness, which may be tested separately in timed trials, the powertrain score reflects how well the Used Engine and transmission work together. Judges look for whether there's enough acceleration from a stop or for passing, if the transmission upshifts smoothly or downshifts without too much delay, and if the Used Engine is smooth and refined or rough and unsophisticated.
Ride Quality
Separate from handling, considerations for ride quality include how it contributes to the vehicle's comfort level. Judges evaluate whether the ride is too firm, too soft, if it feels controlled over bumps and if it's stable on rough roads.
Noise
Judges gauge how much wind, road, Used Engine and external noise enter the cabin during all circumstances, including highway driving and acceleration.
Visibility
Considerations for visibility include whether roof pillars or low roofs obstruct forward visibility, as well as if there are large blind spots to the side or rear. We look for features that can improve visibility, such as rear head restraints that flip down, manually or remotely, and a full-time rearview camera mirror.
Worth the Money?
Judges determine if the test vehicle and the features with which it's equipped are worth the retail price. Considerations apart from the categories above include warranties, free maintenance and whether it has all-wheel drive.
Objective Scoring Categories
Acceleration
Measured testing is logged with a Racelogic Vbox II GPS data logger. Testing takes place at a drag strip where we measure zero-to-60-mph and quarter-mile acceleration. In quarter-mile testing, we use the rollout method, which simulates a drag strip's timing system and how a car is already moving when the timing light beams are triggered — it typically quickens the time by 0.2-0.3 of a second. Rollout is not considered in zero-to-60 mph measurements.
Braking
We measure panic stop braking from 60 mph to simulate how long it takes to stop in an emergency at highway speeds.
Safety and Autonomy
The safety and autonomy component is scored by the number and complexity of safety and driver assistance features equipped on the test vehicle. These include forward collision warning, automatic emergency braking (low versus high speed), lane departure warning (with steering or braking assist), lane-centering steering (low versus high speed), adaptive cruise control (higher speeds only or to a complete stop), blind spot warning, dynamic lines for the backup camera, 360-degree camera systems, rear cross-traffic detection, parking sensors, automatic high beams and adaptive, pivoting headlights.
Fuel Costs
Fuel-cost considerations are based on AAA's national average for fuel prices at the time of publication, gallons used over 12,000 miles a year, the EPA-estimated combined mpg rating and the octane rating recommended for each vehicle's advertised power and fuel-economy ratings.
Child-Safety Seats
's staff includes certified car seat installers who test the fitment of various child-safety seats in our test cars. You can find more information on how we install and test child safety seats in our Car Seat Check section.
's Editorial department is your source for automotive news and reviews. In line with 's long-standing ethics policy, editors and reviewers don't accept gifts or free trips from automakers. The Editorial department is independent of 's advertising, sales and sponsored content departments.