While yes, the Software Advice’s methodologies aren’t great. And no, the Gartner Digital Markets sites, including Software Advice, Capterra, and GetApp, aren’t credible review sites. That doesn’t mean all business software review sites aren’t trustworthy.
Here are some easy ways to tell if online reviews can be trusted:
Reviewers Must Be Product Users
With the rise of fake reviews, we should certainly pay attention to whether reviewers are actual product users. All business software review sites say they have a “vetting process.” However, it seems not all sites require all reviewers to be product users. For example, the Gartner Digital Markets sites do not require product use as a necessity for leaving a review. Trust Radius, G2, and other credible sites actually allow users to upload product screen shots as proof of use. They also require users to log in with a company email address or a LinkedIn account for additional verification.
They Ask Detailed Questions
Many sites are really demand generation or lead generation sites that allow people to leave reviews. These are not actual review sites. The best sites ask the nitty gritty questions to truly get to the heart of the customer experience. This means the reviews ask for large amounts of details including on implementation, training, support, etc. These surveys do a couple things to add credibility – first, how can a non-product user actually answer the “nitty gritty” questions? They can’t. This weeds out non-customers quickly. Second, they provide really valuable information to both buyers and vendors alike. Sites like Trust Radius and IT Central Station do a great job at this!
Methodologies are Reasonable and Easy-to-Understand
The essence of peer reviews is simple. Customers sharing experiences. It’s not quantum physics. It’s not rocket science. It’s simple.
Credible review sites understand this and create comparison metrics and methodologies that are reasonable, easy-to-understand, and are relatively stable. This simplicity does a few things.
Many review sites like G2 and Trust Radius publish their methodologies publicly for anyone to read. They are easy to understand – the gist is quality, quantity, and recency of reviews matters. This is fantastic.
Some sites, though, including the Gartner sites utilize weird, curveball methodologies – at least in my opinion. For example, if I’m reviewing a vendor, I want to see how other organizations like myself are using the product successfully. That means, I want to read reviews or talk to friends at organizations of similar size, industry, etc. Many review sites understand this and have even created breakdown grids specifically for SMB, MM, and ENT comparisons. When looking at these, I would likely assume that the comparisons are reflecting the size of the customers. In most cases, I’d be right. However, Software Advice actually says they way they break down “small vendors” and “enterprise vendors” have nothing to do with the size of the customer base and everything to do with the size of the vendor. Despite their claims that this type of breakdown is “helpful.” It doesn’t help buyers understand the sentiments of users like themselves.
Additionally, stability matters. Research from TrustRadius shows some purchasing decisions can take more than a year and “the process of becoming confident enough to purchase is slowing.” Considering buying cycles take so long and are elongating, having stable methodologies over time that allow for the consistent comparison of vendors is crucial. This will allow buyers to compare apples to apples instead of apples to gorillas, which is what buyers get when they have to review multiple reports with differing methodologies within a single buying cycle. This means that in addition to products changing over time, buyers also need to pay attention to methodology changes over time. Buyers shouldn’t need to review research published more frequently than 6 months to a year. And, buyers should be wary of sites like Peer Insights that adjust methodologies frequently.
They Ask Unbiased Questions
We’ve all heard that language matters and the use of language can sway people to different opinions. Here’s some research on questionnaire design by Pew Research if you’d like to take a look! Needless to say, asking fair and unbiased questions is important.
Be wary of sites that ask about “issues with” certain areas without a counter question in “successes in,” etc. For example, most sites ask about “pros” and “cons” of products. That’s fine. That allows for both good experiences and bad experiences. However, the questions should try to steer clear of using language like “issues” in all the other questions and should simply ask about “experiences in/with” certain areas.
Up until very recently, IT Central Station was one of these sites that had more negatively-leaning survey questions. I’m thrilled they changed the survey to make it more balanced!
All Feedback, No Garbage
Finally, credible review sites differentiate between trashing a business or reviewers and providing feedback. This difference is important. Feedback can be critical, but it provides context and is delivered with credibility. Trashing typically has neither and is harsh. Other forms of “trash” include calling out other reviewers for differences in opinion. When people comment on other reviewers posts and try to discredit them, invalidate them, etc., it makes other reviewers not want to leave reviews. And, it makes vendors not want to work with that company.
For example, sites like Crowd Reviews that allows reviews like this:
Or comments on positive reviews like this:
don’t add any value to the industry or help the company improve. They just tarnish a brand or person’s reputation and make them feel small. This is not what review sites are for. It’s not professional. It has no place in professional services industries. Additionally, the anonymity means the source can’t be verified as an actual product user.
Overall, peer reviews are important to making purchasing decisions and helping vendors improve their products. They are a great space for buyers and customers connecting. And, they are a great space for learning about how customers feel about both you and your competitors.
Although not all online reviews can be trusted, there are definitely review sites that are trustworthy? Find some of those sites and partner away!