How to review Platforms and Platform products: A guide for idiots.
I read a review about the Pixel 3 and Pixel 3 XL from an Apple user. And the article implied that a serious review had taken place and soul searching had happened and that the Pixel 3 was a good phone but failed to "win over" the reviewer for a few reasons.
Now, it isn't exactly unsurprising that such reviews will contain biases and subjectivity. That is unavoidable after all. We're all human (as far as I know) and we all have different circumstances. As a result we'll have tendencies and requirements that aren't necessarily an exact fit for another human being.
This isn't meant to say that you shouldn't have such views. Or even that you should try to eliminate them from your reviews. In fact, I think every review should, as far as the reviewer is capable, disclose any such known tendencies so that other readers can A) understand why certain preferences exist and B) give them a better idea of whether of how they should interpret the results of the review.
No, what this is about are two points that were made in the article which should automatically render the review inadmissible.
The first was a dependency in iMessage and the second is the reviewers degree of immersion in the Apple ecosystem.
People apparently love iMessage. Hey! That is fine. But, it was a foregone conclusion before the reviewer ever touched the competing device that this wouldn't be there. This is a first party application from Apple. And it is Apple's "fault" and not Google's that it is unavailable on Android devices. They don't license it, nor do they publish it to other stores. If this is a breaking criteria for you, you can't actually claim to have ever taken this seriously. There was never any chance.
It would be like reviewing a katana as a replacement weapon for a gun even though having a device that fires bullets is a key requirement for you. I don't need a review katana to find out it won't fire bullets. Nor does pointing out that I can't accept the katana as a replacement bullet firing weapon help ANYONE at all. It is a totally useless and un-necessarily biased inclusion.
If you feel the absolute need to make such inclusions. Do so in your conclusions. Not in your main points. You might say something like, "... so I find the Katana to be an excellent weapon for anyone interested in close combat weaponry or exquisite decorative pieces. I can respect it, but as a game hunter, I need a ranged weapon for take down and maybe a smaller blade for preparing the carcass".
Note how I don't let my own preferences and requirements tarnish the overall review? Mind blowing I know! And, as an added bonus, I explained what it is about my situation which drives my own decision not to buy one which might help others in a similar decision decide. And since I like this fictional katana, I went to suggest some situations under which it might be good.
The next problem is the reviewer's own investment in another ecosystem. This is even crazier to include in a review as it doesn't actually have ANYTHING to do with the product being reviewed. Back to the katana review as an example, it would be like saying "unfortunately I have a gun cabinet, holsters, gun cases and ammo which are incompatible with this katana".
Once again, the reviewer KNEW going into this that the Android phone was NOT an Apple product. But, even worse, this isn't even a first party feature or even the availability of a 3rd party app or anything valuable like that to include. This is an, "I've spent a sickening amount of money on Apple related stuff and I am as such, bound to this fruity world".
How would I have approached such a review? I would start by reviewing the device on it's own. I may reach to competition to help rate common features like the quality of the camera, or compare app/game performance on some title which is on both. I might talk about the ecosystem if there are direct benefits.
On topics like iMessage, I would compare it to the competition in terms of features. Claiming "I can't live without iMessage" says nothing about why. Which specific features are missing? Are there ways around this? If there are specific things missing, it would help someone make a decision. Just because you can't live without feature A, doesn't mean it is important to someone else.
On the topic of ecosystem integration, it is definitely worth pointing out. Just not as a part of the review of the device. Some people may not know or may not have thought about the potential need to re-purchase certain apps or whether or not they are available. They may not know that certain features only work with certain devices. But, once again, not everyone has the same level of investment in the eco system. For instance, an Apple user, whose only device is an iPhone and who hasn't purchased much if anything from the store has little to nothing tying them to ecosystem.
I also feel like people sell up the cost of leaving an eco system WAY too much. I moved off of Windows Phone to Android. I had made quite a few purchases in the Windows Store. But, the reality of the situation was that many of those purchases were in-game purchases or apps I didn't use any longer. There were maybe 2-5 paid apps I was actively using when I switched. My media was all DRM free as is Apple's. My streaming services were available on the other system and alternatives existed if there were issues.
After dropping $1k on a phone... spending $25 to buy back apps... not really a problem. And, with a little patience you can probably find a deal which includes a store credit on the purchase of a device.
This isn't to say that there aren't people who are in DEEP, or who have integrations which they can't live without. But, let's be fair... those people aren't reading these reviews. Or, if they are, they are just reading them for the sake of reading.
Now, it isn't exactly unsurprising that such reviews will contain biases and subjectivity. That is unavoidable after all. We're all human (as far as I know) and we all have different circumstances. As a result we'll have tendencies and requirements that aren't necessarily an exact fit for another human being.
This isn't meant to say that you shouldn't have such views. Or even that you should try to eliminate them from your reviews. In fact, I think every review should, as far as the reviewer is capable, disclose any such known tendencies so that other readers can A) understand why certain preferences exist and B) give them a better idea of whether of how they should interpret the results of the review.
No, what this is about are two points that were made in the article which should automatically render the review inadmissible.
The first was a dependency in iMessage and the second is the reviewers degree of immersion in the Apple ecosystem.
People apparently love iMessage. Hey! That is fine. But, it was a foregone conclusion before the reviewer ever touched the competing device that this wouldn't be there. This is a first party application from Apple. And it is Apple's "fault" and not Google's that it is unavailable on Android devices. They don't license it, nor do they publish it to other stores. If this is a breaking criteria for you, you can't actually claim to have ever taken this seriously. There was never any chance.
It would be like reviewing a katana as a replacement weapon for a gun even though having a device that fires bullets is a key requirement for you. I don't need a review katana to find out it won't fire bullets. Nor does pointing out that I can't accept the katana as a replacement bullet firing weapon help ANYONE at all. It is a totally useless and un-necessarily biased inclusion.
If you feel the absolute need to make such inclusions. Do so in your conclusions. Not in your main points. You might say something like, "... so I find the Katana to be an excellent weapon for anyone interested in close combat weaponry or exquisite decorative pieces. I can respect it, but as a game hunter, I need a ranged weapon for take down and maybe a smaller blade for preparing the carcass".
Note how I don't let my own preferences and requirements tarnish the overall review? Mind blowing I know! And, as an added bonus, I explained what it is about my situation which drives my own decision not to buy one which might help others in a similar decision decide. And since I like this fictional katana, I went to suggest some situations under which it might be good.
The next problem is the reviewer's own investment in another ecosystem. This is even crazier to include in a review as it doesn't actually have ANYTHING to do with the product being reviewed. Back to the katana review as an example, it would be like saying "unfortunately I have a gun cabinet, holsters, gun cases and ammo which are incompatible with this katana".
Once again, the reviewer KNEW going into this that the Android phone was NOT an Apple product. But, even worse, this isn't even a first party feature or even the availability of a 3rd party app or anything valuable like that to include. This is an, "I've spent a sickening amount of money on Apple related stuff and I am as such, bound to this fruity world".
How would I have approached such a review? I would start by reviewing the device on it's own. I may reach to competition to help rate common features like the quality of the camera, or compare app/game performance on some title which is on both. I might talk about the ecosystem if there are direct benefits.
On topics like iMessage, I would compare it to the competition in terms of features. Claiming "I can't live without iMessage" says nothing about why. Which specific features are missing? Are there ways around this? If there are specific things missing, it would help someone make a decision. Just because you can't live without feature A, doesn't mean it is important to someone else.
On the topic of ecosystem integration, it is definitely worth pointing out. Just not as a part of the review of the device. Some people may not know or may not have thought about the potential need to re-purchase certain apps or whether or not they are available. They may not know that certain features only work with certain devices. But, once again, not everyone has the same level of investment in the eco system. For instance, an Apple user, whose only device is an iPhone and who hasn't purchased much if anything from the store has little to nothing tying them to ecosystem.
I also feel like people sell up the cost of leaving an eco system WAY too much. I moved off of Windows Phone to Android. I had made quite a few purchases in the Windows Store. But, the reality of the situation was that many of those purchases were in-game purchases or apps I didn't use any longer. There were maybe 2-5 paid apps I was actively using when I switched. My media was all DRM free as is Apple's. My streaming services were available on the other system and alternatives existed if there were issues.
After dropping $1k on a phone... spending $25 to buy back apps... not really a problem. And, with a little patience you can probably find a deal which includes a store credit on the purchase of a device.
This isn't to say that there aren't people who are in DEEP, or who have integrations which they can't live without. But, let's be fair... those people aren't reading these reviews. Or, if they are, they are just reading them for the sake of reading.
Comments
Post a Comment