F-Secure secured the Award for the worst on AV-Comparatives Enhanced Real-World Test 2020 – Consumer
Accepted Answers
-
-
Hello,
I enjoyed reading AV-Comparatives website about this research. A good description and reasoning of their test ways, meanings and importance. And I think that this is generally a fair result (for all involved).
But..
F-Secure secured the Award for the worst on AV-Comparatives Enhanced Real-World Test 2020 – Consumer
is actually about the Award for "Standard (ATP Consumer)". Although this was indeed the worst result, it was only among the seven (six) companies that took part. And where no company showed "the worst" (non certified) result.
And..
F-Secure couldn't score decently because its wasn't good in any given test
is actually, was excellent in five and good in one given test.
On the other hand, the difference with the rest of the participants is - of course - dramatic.
So, as already mentioned above, the reasons for this are probably being sorted out (or have already been clarified). Also, as stated in in aftermath of test:
After the test, we provide each participating vendor with sufficient data to assist them in understanding any of their missed test cases.
Anyway, perhaps, it is matter of too obfuscated ways of some tricks (though it is meaning of APT). At least, my amateur attempt to repeat one of not passed cases (based on its short description) led to detection by F-Secure. But this is without any matured disguises and no "technical" ways of distribution.
Also I like their conclusion:
In our opinion, the goal of every AV/EPP/EDR system should be to detect and prevent attacks or other malware as soon as possible. In other words, if the attack is detected before, at or soon after execution, thus preventing e.g. the opening of a Command and Control Channel, there is no need to prevent post-exploitation activities. A good burglar alarm should go off when somebody breaks into your house, not wait until they start stealing things.
Also some things that are not entirely clear or are in some way mitigating points (my opinion):
- what exactly was false positive test. I mean, just scripts (and so) as such (safe, normal ones). Or obfuscated (but safe) or suspicious (but normal) involved too.
- was POST detection always at a measurable stage? Or how far post-exploitation went (without prior detection).
- sounds that is hugely based on non-modified(?) "Meterpreter / PowerShell Empire" pieces. I encountered how some of tools overly opposed to this and others are more loyal due to educational use as well. And in another sense - alarming at the slightest hint of the presence of use these (no one does this, perhaps).
- an admin account is targeted. But there are more and more recommendations to use a standard account (restricted rights), even for home users. This is of course a bonus for any security software... but it would just be interesting to see what the results would be when using a non-admin account (and in general the cases used would be applicable or not). But, yes, for a more realistic set-up everything was done right.
Thanks! And sorry for my reflections.
Answers
-
Hi @ARAED
Thank you for sharing this with us.
There are several independent review sites that carry out series of tests and they decide on who and what to test.
These tests occur multiple times annually on a couple different sites, and generally speaking we have scored extremely well. For example, in their previous malware test we got top scores - https://www.av-comparatives.org/tests/malware-protection-test-september-2020/
Cheers!
-
F-Secure indeed was the worst performer.
I read the whole report, Bitdefender blocked most threats even before running, but F-Secure blocked most after running and even block didn't block some of the scripts or viruses according to AV-Comparatives.
This community is for raising the voice of consumers, F-Secure shouldn't have banned me after seeing the post
I can raise my voice, I have a three year of F-Secure safe subscription, I won't hesitate to take matters that concerns the security of my pc and smartphone to this forum
-
Hello,
Sorry for my reply! Just as a discussion between Community users.
This community is for raising the voice of consumers, F-Secure shouldn't have banned me after seeing the post
I can raise my voice, I have a three year of F-Secure safe subscription, I won't hesitate to take matters that concerns the security of my pc and smartphone to this forum
I don't think they were going to ban you of the reason like "seeing this the post / subject of post". I think that the automatic triggering of the platform engine was more likely. F-Secure Community is probably still powered by (vanillaforums.com) platform.
For example, some specific triggers, the presence of pictures or URLs, attached files or excessive editing of replies. Or something else. Maybe because of other topics or replies. Or anything else that the platform might regard as suspicious. In addition, they unbanned you (if there really was a ban). So... perhaps this is an accident.
I read the whole report
Yes, me too.
Their formulations about results are, in principle, quite accurate, but leave room for speculation. By any side / matter.
This includes the importance and difference of detections pre/on/post, the "reason" for detections at one stage or another. And so on.
Anyway, this is about only "Advanced persistent threat" concept and many things are quite obscure. I described my own perception in the previous reply.
To block something before running is good - with your mentioned example ("before the threat was executed, due to heuristics for malicious scripts.") So, does it mean by scanning the first involved link, or directly "blocking" the final malicious payload that comes into play, as real-time protection? If it was "PRE" - what is meant by malicious scripts? if it is so clear to understand why others have not had such success.
Simply put, what is the difference that the user will run the script with the download of something, albeit "dubious" or the establishment of a connection to some server. And if it is done with fraudulent intent without the user's knowledge?
So.. I liked the results of the other two leaders (of the test) better. They look more natural.
Well, and as stated on the website:
From our experience, we know that many consumer AV programs do not provide effective protection against the types of threat used in this test. For this reason, a consumer security app that detects even 5 out of 15 threats is worthy of an award for “Advanced Threat Protection” (ATP).
And each of the seven companies (in a sense, six; AVG/Avast) received an award. F-Secure detected 6 of 15 threats. So, "Standard". And as the F-Secure Community Moderator pointed out, the 'general' recent tests were with quite well results (top). I also read recent third party reviews about some of F-Secure's advanced detection capabilities (where there were different meanings, but I was satisfied - because it was a great discussion and understanding of the reasons for the detection or not). Some of the heuristic engines are awesome. But too often they can be alarmed for no reason (not so much to react to a 'normal' file or process, but can make people nervous about locking a safe file due to an overly generic approach). In part, this situation with the "ban" you described is also something like this.
In any case, it will be interesting to see the results of future tests on ATP. And the results of F-Secure (if they participate).
I feel about the current ones as about fair ones. And at the same time, they are not completely demonstrative. It's just that everyone can make certain conclusions based on their own desires.
Thanks!
-
Hi @ARAED
The community is open to every customer, current and prospective, to voice out and we listen to every view.
We have moderation happening round the clock which was why your account was released as soon as it was detected to have been banned unexpectedly. As you might know, we have been doing some migration, so a few glitches here are there are sometimes unavoidable.
We will take this as a learning point!
We thank you for your understanding and we'll keep an eye as usual to ensure we keep improving on this and more.
Cheers!
-
-
