Master caution - two customers report FSAV ESS "failed to scan mail" possible wrong update in AUA/AU

etomcat

Hello,

 

Possible master caution incident - two independet hungarian customers suddenly report FSAV ESS "failed to scan mail" possible wrong update in AUA/AUS?

 

One of them: non-clustered, Win2008 R2 Std + Exch 2010, FSAV for Exchange 11.01 build 157, first e-mail arrived in the quarantine today at 13:07 CEST, 900 alerts generated on ~150 incoming mails in ~90 mins, some of the alerts, getting stalled also generated further alerts in a chain reaction.

 

A diag is available in F-Secure's incoming FTP folder.

 

Yours Sincerely: Tamas Feher, 2F 2000 Kft., Hungary.

etomcat

Dear F-Secure Partner Support,

 

Please find a diagnostic output at: (censored)

 

Yours Sincerely: Tamas Feher, 2F 2000 Kft., Hungary.

etomcat

Hello,

 

A third hungarian customer reports the FSAV ESS downtime is possibly related to GEMINI error messages and that reboot doesn't help.

 

Yours Sincerely: Tamas Feher, 2F 2000, Hungary.

BUSTER76

Hi There,

 

same problem here since Gemini V3.2.384 in Germany.

 

Diag is on the way to f-secure.

Disabled Gemini Module, Email is working again so far...

 

Waiting for solution...

 

buster

etomcat

Dear Buster,

 

Thanks for the clue!

 

> Disabled Gemini Module

 

Is that only possible if the FSAV ESS computer is under Policy Manager control, I think? But most are stand-alone here (probably for fear of misconfiguration if lumped together with other endpoints in PMC).

 

Yours Sincerely: Tamas Feher, 2F 2000 Kft., Hungary.

 

 

BUSTER76

Hi,

 

You can do that in the ESS Webinterface.

(I dont think PM Control is necessary... but i'm not sure.)

I dont know the the english Option, could be "common", below that there is a "Module" Option where you can see the 3 Scanengines.

In the settings there you can disable the Engine...

 

TimoFS

Let me start by sincerely apologizing for any inconveniences or problems that this issue has caused you.

 

This problem was introduced in Gemini 2015-09-24_01 update published on 24th of September 2015 at 13:18 (EEST) and promptly fixed in Gemini 2015-09-24_03  update published on 24th of September 2015 at 16:06 PM (EEST). We have initiated a root cause analysis process to identify how we can further improve our processes and automation to avoid similar incidents in the future.

 

If you have any further questions or feedback please do not hesitate to contact F-Secure at any time

etomcat

Hello,

 

(I think the  following advice is now depreciated, since the fixed Gemini signature updates have been already published.)

 

I got an e-mail from F-Secure, advising this setting may partially cure the problem, until a new bugfixed Gemini update can be published.

 

Best regards: Tamas Feher, Hungary.

 

********************************************

 

HairyDawg

Disabling the Gemini engine has worked for me. Thanks Buster76!

etomcat

Dear Timo,

 

I just ran an FSAUA-reset, but even after that I'm still receiving Gemini_today_01 from fsbwserver.f-secure.com, in FSPM 12.00's AUA/AUS?

 

Yours Sincerely: Tamas Feher.

BUSTER76

Hello,

 

since then i didnt get any gemini update, too....

Aquarius Update 2015-09-24_06 is the latest i received...

 

Buster

TimoFS

It takes some time for the update to propagate to all the different update servers that we have. Unfortunately you just need to give it some time... The good news is that all the products that have Gemini use the exact same update channel to receive the update, and I can confirm that the update *is* in the channel. I understand time is of the essence but in this case all you can do is wait. You can try checking for the update every now and then. It will become available as soon as it has propagated to all our update servers. Thank you for your patience and understanding.

HairyDawg

I have received the update v3.2.384 and can confirm that messages are flowing now. 

BUSTER76

Hi Timo,

 

thanks for that Info!

etomcat

Hello,

 

My test system has just downloaded these from fsbwserver.f-secure.com:

 

F-Secure Aquarius Update 2015-09-24_07

and

F-Secure Gemini Update 2015-09-24_03

 

Yours Sincerely: Tamas Feher, 2F 2000 Kft., Hungary.

BUSTER76

F-Secure Gemini Update 2015-09-24_03 just arrived,

will activate it tomorrow morning again...

 

 

thx

etomcat

Dear Timo,

 

> It takes some time for the update to propagate to all the different update servers that we have. Unfortunately you just need to give it some time.

 

I would have expected F-Secure Corp. to issue a central command via the ORSP cloud, to instantly neutralize the Gemini scan engine until the fixed 24_03 signature update becomes not just released but practically available to end user's computers. We have already seen the cloud quelling false malware alarms instantly and very effectively!

 

Best Regards: Tamas Feher.

etomcat

Dear F-Secure Partner Support,

 

Is it normal that I see so many of these error messages in FSPM 12.00 AUA/AUS?

 

[ 3000] Fri Sep 25 14:02:03 2015 (3):

Installation of 'F-Secure Aquarius Update 2015-09-25_04' : Failed, will retry

 

In the GUI, both AquaWin32 and AquaLNX32 are marked as "failed, will retry".

 

This is unsual, because it was nomal during the previous days, for example:

[ 5112] Thu Sep 24 17:12:09 2015 (3):

Installation of 'F-Secure Aquarius Update 2015-09-24_07' : Success

 

Yours Sincerely: Tamas Feher, Hungary.