/ BLOG / Exchange server w3wp high memory/cpu
This post is largely for future me. I’m fed up with (re)writing/(re)discovering some of these queries. However, I also hope it can help other people.
This post was written specifically whilst I was finishing up with an Exchange 2010 installation. However, should work verbatim with 2007 and some of the queries may require a little alteration for 2013. If you’re still on 2003. I’m sorry.
So your Exchange server has a w3wp instance with high memory and cpu.
If you’re on 2010, ensure that you’re on a patch level that covers the issue described in KB2800133.
First step is to find out what the instance is running. Use task manager to show the full command line of the instance. Now check the Windows Event logs. Is there anything interesting? If not move on.
Try recycling that AppPool instance. If that doesn’t help long term then we need to start analysing logs.
If you find that it’s the MSExchangePowerShellAppPool, there’s probably just a console open somewhere doing a lot of talking, or recently having done a lot of talking. It’ll sort itself out shortly.
If it’s the MSExchangeSyncAppPool then the odds are likely good that you have a problem device. To figure out which, make sure that IIS is logging access. If it’s not, wait a day. Or at least a few hours if you can’t.
Now, run the IIS logs through logparser with the following query -
SELECT TOP 500 TO_TIMESTAMP(TO_DATE(date), TO_TIME(time)) as Time, cs-username as User, cs(user-agent) as DeviceID, TO_INT(EXTRACT_PREFIX(EXTRACT_SUFFIX(cs-uri-query, 0, '_RpcC'), 0, '_')) As RPCCount, sc-status as Status, sc-substatus as SubStatus, sc-bytes as Bytes, DIV(sc-bytes, 1024) AS KBytes, time-taken, DIV(time-taken, 1000) as Seconds, cs-uri-query FROM 'path\to\log\files\*.log' WHERE RPCCount >= 1500 AND cs-uri-query LIKE '%Cmd=Sync%' AND cs-uri-query LIKE '%Ty:Co%' ORDER BY Bytes DESC
If you find a user frequently popping up to the top, it’s likely their device causing the problem. Disable their ActiveSync privileges, recycle the AppPool and see how things fair. Repeat as necessary.
If you find it’s a specific user, but you cannot “fix” their device, throttle their device instead, using a throttling policy.
If you find you’re not getting anywhere then start looking for unusually high number of requesting devices+users -
SELECT TOP 500 cs-username AS User, cs(User-Agent) AS DeviceType, COUNT(*) as Hits FROM 'path\to\log\files\*.log' WHERE cs-uri-stem LIKE '%Microsoft-Server-ActiveSync%' GROUP BY User, DeviceType ORDER BY Hits, DeviceType DESC
If it’s the MSExchangeOWAAppPool then you may have someone attempting to log into an account. It should be locking out if they’ve found a real account.
SELECT TOP 500 c-ip AS IP, cs(User-Agent) AS DeviceType, COUNT(*) as Hits FROM 'path\to\log\files\*.log' WHERE cs-uri-stem LIKE '%/OWA%' GROUP BY IP, DeviceType ORDER BY Hits, DeviceType DESC
If you’re still not getting anywhere, revisit the Windows Event Logs. Check that there’s nothing showing up in there that’s relevant. If there really isn’t anything then start cutting down the problem.
Try to isolate your Exchange’s CAS from the internet temporarily. Does it quieten down? If not isolate them/it from the LAN. Does it quieten down? Start looking at the logs in different ways.