{"id":1319,"date":"2022-01-04T18:19:57","date_gmt":"2022-01-05T00:19:57","guid":{"rendered":"http:\/\/zewwy.ca\/?p=1319"},"modified":"2022-01-04T18:19:57","modified_gmt":"2022-01-05T00:19:57","slug":"email-stuck-in-exchange-transport-in-2022","status":"publish","type":"post","link":"https:\/\/zewwy.ca\/index.php\/2022\/01\/04\/email-stuck-in-exchange-transport-in-2022\/","title":{"rendered":"Email Stuck in Exchange Transport in 2022"},"content":{"rendered":"<p>Happy New Year!<\/p>\n<p>If you are an exchange admin you may want to check out the notice from Microsoft. But you probably already have considering it started in the beginning of the new year: <a href=\"https:\/\/techcommunity.microsoft.com\/t5\/exchange-team-blog\/email-stuck-in-exchange-on-premises-transport-queues\/ba-p\/3049447\">Email Stuck in Exchange On-premises Transport Queues &#8211; Microsoft Tech Community<\/a><\/p>\n<p>So you probably already implemented this fix.<\/p>\n<p>We have now created a solution to address the problem of messages stuck in transport queues on Exchange Server 2016 and Exchange Server 2019 because of a latent date issue in a signature file used by the malware scanning engine within Exchange Server.\u00a0<strong>Customer action is required to implement this solution.<\/strong>\u00a0When the issue occurs, you\u2019ll see errors in the Application event log on the Exchange Server, specifically event 5300 and 1106 (FIPFS), as illustrated below:<\/p>\n<pre>Log Name: Application \r\nSource: FIPFS \r\nLogged: 1\/1\/2022 1:03:42 AM \r\nEvent ID: 5300 \r\nLevel: Error \r\nComputer: server1.contoso.com\r\nDescription: The FIP-FS \"Microsoft\" Scan Engine failed to load. PID: 23092, Error Code: 0x80004005. Error Description: Can't convert \"2201010001\" to long.<\/pre>\n<pre>Log Name: Application \r\nSource: FIPFS \r\nLogged: 1\/1\/2022 11:47:16 AM \r\nEvent ID: 1106 \r\nLevel: Error \r\nComputer: server1.contoso.com \r\nDescription: The FIP-FS Scan Process failed initialization. Error: 0x80004005. Error Details: Unspecified error.<\/pre>\n<h2 id=\"toc-hId-333878005\"><span class=\"ez-toc-section\" id=\"Using_the_Automated_Solution\"><\/span><span style=\"font-size: xx-large;\">Using the Automated Solution<\/span><span class=\"ez-toc-section-end\"><\/span><\/h2>\n<ul>\n<li>Download the script here:\u00a0<a href=\"https:\/\/aka.ms\/ResetScanEngineVersion\" target=\"_blank\" rel=\"noopener noreferrer\">https:\/\/aka.ms\/ResetScanEngineVersion<\/a><\/li>\n<li>Before running the script, change the execution policy for PowerShell scripts by running\u00a0<strong>Set-ExecutionPolicy -ExecutionPolicy RemoteSigned<\/strong>.<\/li>\n<li>Run the script on each Exchange mailbox server that downloads antimalware updates in your organization (use elevated Exchange Management Shell).<\/li>\n<\/ul>\n<p>Edge Transport servers are unaffected by this issue. You can run this script on multiple servers in parallel. After the script has completed, you will see the following output:<\/p>\n<pre>[PS] C:\\Program Files\\Microsoft\\Exchange Server\\V15\\Scripts&gt;.\\Reset-ScanEngineVersion.ps1\r\nEXCH1 Stopping services...\r\nEXCH1 Removing Microsoft engine folder...\r\nEXCH1 Emptying metadata folder...\r\nEXCH1 Starting services...\r\nWARNING: Waiting for service 'Microsoft Filtering Management Service (FMS)' to start...\r\nWARNING: Waiting for service 'Microsoft Filtering Management Service (FMS)' to start...\r\nWARNING: Waiting for service 'Microsoft Filtering Management Service (FMS)' to start...\r\nWARNING: Waiting for service 'Microsoft Filtering Management Service (FMS)' to start...\r\nWARNING: Waiting for service 'Microsoft Exchange Transport (MSExchangeTransport)' to start...\r\nEXCH1 Starting engine update...\r\nRunning as EXCH1-DOM\\Administrator.\r\n--------\r\nConnecting to EXCH1.CONTOSO.com.\r\nDispatched remote command. Start-EngineUpdate -UpdatePath <a href=\"http:\/\/amupdatedl.microsoft.com\/server\/amupdate\" target=\"_blank\" rel=\"noopener noreferrer\">http:\/\/amupdatedl.microsoft.com\/server\/amupdate<\/a>\r\n--------\r\n[PS] Add-PSSnapin Microsoft.Forefront.Filtering.Management.Powershell.\r\n--------\r\n[PS] C:\\Program Files\\Microsoft\\Exchange Server\\V15\\Scripts&gt;Get-EngineUpdateInformation<strong>\r\n<\/strong><strong>\r\nEngine \u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 : Microsoft<\/strong>\r\n<strong>LastChecked <\/strong>\u00a0 \u00a0     \u00a0\u00a0<strong>: 01\/01\/2022 08:58:22 PM -08:00<\/strong>\r\n<strong>LastUpdated <\/strong>\u00a0 \u00a0 \u00a0\u00a0    <strong>: 01\/01\/2022 08:58:31 PM -08:00<\/strong>\r\n<strong>EngineVersion \u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 : 1.1.18800.4<\/strong>\r\n<strong>SignatureVersion \u00a0\u00a0\u00a0\u00a0 : 1.355.1227.0<\/strong>\r\n<strong>SignatureDateTime \u00a0\u00a0\u00a0 : 01\/01\/2022 03:29:06 AM -08:00<\/strong>\r\n<strong>UpdateVersion \u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 : 2112330001 (note: higher version number starting with 211233... is also OK)<\/strong>\r\n<strong>UpdateStatus <\/strong>\u00a0\u00a0\u00a0 \u00a0\u00a0\u00a0\u00a0\u00a0<strong>: UpdateAttemptSuccessful<\/strong><\/pre>\n<h2 id=\"toc-hId--1473576458\"><span class=\"ez-toc-section\" id=\"Using_the_Manual_Solution\"><\/span><span style=\"font-size: xx-large;\">Using the Manual Solution<\/span><span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p>In lieu of using the script, customers can also manually perform steps to resolve the issue and restore service. To manually resolve this issue, you must perform the following steps on each Exchange mailbox server in your organization that downloads antimalware updates. Edge Transport servers are unaffected by this issue.<\/p>\n<p><strong>Verify the impacted version is installed<br \/>\n<\/strong>Run Get-EngineUpdateInformation and check the UpdateVersion information. If it starts with &#8220;22&#8230;&#8221; then proceed. If the installed version starts with &#8220;21&#8230;&#8221; you do not need to take action.<\/p>\n<p><strong>Remove existing engine and metadata<\/strong><br \/>\n1. Stop the Microsoft Filtering Management service.\u00a0 When prompted to also stop the Microsoft Exchange Transport service, click Yes.<br \/>\n2. Use Task Manager to ensure that updateservice.exe is not running.<br \/>\n3. Delete the following folder: %ProgramFiles%\\Microsoft\\Exchange Server\\V15\\FIP-FS\\Data\\Engines\\amd64\\Microsoft.<br \/>\n4. Remove all files from the following folder: %ProgramFiles%\\Microsoft\\Exchange Server\\V15\\FIP-FS\\Data\\Engines\\metadata.<\/p>\n<p><strong>Update to latest engine<\/strong><br \/>\n1. Start the Microsoft Filtering Management service and the Microsoft Exchange Transport service.<br \/>\n2. Open the Exchange Management Shell, navigate to the Scripts folder (%ProgramFiles%\\Microsoft\\Exchange Server\\V15\\Scripts), and run\u00a0<em>Update-MalwareFilteringServer.ps1 &lt;server FQDN&gt;<\/em>.<\/p>\n<p><strong>Verify engine update info<\/strong><br \/>\n1. In the Exchange Management Shell, run\u00a0<em>Add-PSSnapin Microsoft.Forefront.Filtering.Management.Powershell<\/em>.<br \/>\n2. Run\u00a0<em>Get-EngineUpdateInformation<\/em>\u00a0and verify the UpdateVersion information is 2112330001 (or higher)<\/p>\n<p>After updating the engine, we also recommend that you verify that mail flow is working and that FIPFS error events are not present in the Application event log.<\/p>\n<p>If you want to know why this happened here&#8217;s a answer from the comments:<\/p>\n<p>John_C_Kirk &#8211; &#8220;This wasn&#8217;t due to a change on 31st Dec. The problem is caused by an integer overflow error: the anti-malware component is converting the date\/time into &#8220;YYMMDDHHMM&#8221; format and storing it as a signed 32-bit number (max value\u00a02147483648). So, in Dec 2021, the number would start with &#8220;2112&#8230;&#8221; (below the threshold). In Jan 2022, the number would start with &#8220;2201&#8230;&#8221; (above the threshold).&#8221;<\/p>\n<p>Two Thumbs up on implementation.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Happy New Year! If you are an exchange admin you may want to check out the notice from Microsoft. But you probably already have considering it started in the beginning of the new year: Email Stuck in Exchange On-premises Transport Queues &#8211; Microsoft Tech Community So you probably already implemented this fix. We have now &hellip; <\/p>\n<p class=\"link-more\"><a href=\"https:\/\/zewwy.ca\/index.php\/2022\/01\/04\/email-stuck-in-exchange-transport-in-2022\/\" class=\"more-link\">Continue reading<span class=\"screen-reader-text\"> &#8220;Email Stuck in Exchange Transport in 2022&#8221;<\/span><\/a><\/p>\n","protected":false},"author":3,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"sfsi_plus_gutenberg_text_before_share":"","sfsi_plus_gutenberg_show_text_before_share":"","sfsi_plus_gutenberg_icon_type":"","sfsi_plus_gutenberg_icon_alignemt":"","sfsi_plus_gutenburg_max_per_row":"","footnotes":""},"categories":[36],"tags":[397,381,398],"class_list":["post-1319","post","type-post","status-publish","format-standard","hentry","category-exchange","tag-mail-flow","tag-stuck","tag-transport"],"_links":{"self":[{"href":"https:\/\/zewwy.ca\/index.php\/wp-json\/wp\/v2\/posts\/1319","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/zewwy.ca\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/zewwy.ca\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/zewwy.ca\/index.php\/wp-json\/wp\/v2\/users\/3"}],"replies":[{"embeddable":true,"href":"https:\/\/zewwy.ca\/index.php\/wp-json\/wp\/v2\/comments?post=1319"}],"version-history":[{"count":1,"href":"https:\/\/zewwy.ca\/index.php\/wp-json\/wp\/v2\/posts\/1319\/revisions"}],"predecessor-version":[{"id":1320,"href":"https:\/\/zewwy.ca\/index.php\/wp-json\/wp\/v2\/posts\/1319\/revisions\/1320"}],"wp:attachment":[{"href":"https:\/\/zewwy.ca\/index.php\/wp-json\/wp\/v2\/media?parent=1319"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/zewwy.ca\/index.php\/wp-json\/wp\/v2\/categories?post=1319"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/zewwy.ca\/index.php\/wp-json\/wp\/v2\/tags?post=1319"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}