![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(64, 23%, 16%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EAH%3C/text%3E%3C/svg%3E)
Windows 10
A Microsoft operating system that runs on personal computers and tablets.
11,981 questions
Hello All,
My suggestion that killing all instances of EpicWebHelper.exe before entering hibernation might be an effective measure against this bug check probably sounds rather implausible. In case it is of any use to someone, here is the (rather lengthy) explanation of why I came to that conclusion.
INTERNAL_POWER_ERROR (a0)
The power policy manager experienced a fatal error.
Arguments:
Arg1: 00000000000000f0, The system failed to complete (suspend) a power transition in a timely manner.
Arg2: 0000000000000004, The system power state in transition. 0x4 = PowerSystemSleeping3
Arg3: 000000000000000e, The sleep checkpoint most recently reached. 0xE = PopSleepCheckpointSuspendDriversBefore
Arg4: ffffaa094b147040, A pointer to the thread currently processing the request.
The stack of the thread that initiated the bug check is:
nt!KeBugCheckEx
nt!PopPowerActionWatchdog+0xa5
nt!KiProcessExpiredTimerList+0x172
nt!KiRetireDpcList+0x5dd
nt!KiIdleLoop+0x9e
The stack of the thread processing the request (Arg4) is:
nt!KiSwapContext+0x76
nt!KiSwapThread+0x500
nt!KiCommitThreadWait+0x14f
nt!KeWaitForSingleObject+0x233
nt!PopEndMirroring+0x91
nt!MmDuplicateMemory+0x1f7
nt!PopTransitionToSleep+0x135
nt!PspSystemThreadStartup+0x55
nt!KiStartSystemThread+0x28
The “Wait Start TickCount” of this thread is 7622 and the TickCount at the time of the crash is 45937; (45937 – 7622) / 64 = 598.7 seconds. The PopPowerActionWatchdog is set at 600 seconds; the crash happens 10 minutes after initiating hibernation.
The stack of the “ActionWorkerThread” (POP_POWER_ACTION.ActionWorkerThread, !poaction) is:
nt!KiSwapContext+0x76
nt!KiSwapThread+0x500
nt!KiCommitThreadWait+0x14f
nt!KeWaitForSingleObject+0x233
nt!ExpWaitForResource+0x6d
nt!ExAcquireResourceExclusiveLite+0x1fe
nt!ExAcquireTimeRefreshLock+0x22
nt!PoBroadcastSystemState+0x310
nt!PopSetDevicesSystemState+0x87
nt!PopTransitionSystemPowerStateEx+0xbfe
nt!NtSetSystemPowerState+0x4c
nt!KiSystemServiceCopyEnd+0x25 (TrapFrame @ ffffa90d`eaba87c0)
nt!KiServiceLinkage
nt!PopIssueActionRequest+0x216
nt!PopPolicyWorkerAction+0x79
nt!PopPolicyWorkerThread+0x94
nt!ExpWorkerThread+0x105
nt!PspSystemThreadStartup+0x55
nt!KiStartSystemThread+0x28
This thread is blocked, waiting for the “TimeRefresh” lock. Examining that lock shows:
Resource @ nt!ExpTimeRefreshLock (0xfffff8024ce194c0) Exclusively owned
Contention Count = 24
NumberOfExclusiveWaiters = 14
Threads: ffffaa094e32f580-01<*>
THREAD ffffaa094e32f580 Cid 2fac.329c Teb: 0000002ad2aed000 Win32Thread: 0000000000000000 WAIT: (Executive) KernelMode Non-Alertable
ffffa90df0b16a18 NotificationEvent
IRP List:
ffffaa094dd9c570: (0006,04c0) Flags: 00060043 Mdl: ffffaa094e5d25b0
Not impersonating
DeviceMap ffffbd8220944db0
Owning Process ffffaa0950216080 Image: EpicWebHelper.exe
Attached Process N/A Image: N/A
Wait Start TickCount 8273 Ticks: 37664 (0:00:09:48.500)
Context Switch Count 14 IdealProcessor: 13
UserTime 00:00:00.000
KernelTime 00:00:00.000
Win32 Start Address 0x00007ffaaf11f460
Stack Init ffffa90df0b17bd0 Current ffffa90df0b16200
Base ffffa90df0b18000 Limit ffffa90df0b11000 Call 0000000000000000
Priority 12 BasePriority 8 PriorityDecrement 48 IoPriority 2 PagePriority 5
Child-SP RetAddr Call Site
ffffa90d`f0b16240 fffff802`4c4130b0 nt!KiSwapContext+0x76
ffffa90d`f0b16380 fffff802`4c4125df nt!KiSwapThread+0x500
ffffa90d`f0b16430 fffff802`4c411e83 nt!KiCommitThreadWait+0x14f
ffffa90d`f0b164d0 fffff802`4e8f38b8 nt!KeWaitForSingleObject+0x233
ffffa90d`f0b165c0 fffff802`4e8dce2f Ntfs!NtfsWaitOnIo+0x84
ffffa90d`f0b16610 fffff802`4e8d88d0 Ntfs!NtfsNonCachedIo+0x9bf
ffffa90d`f0b168a0 fffff802`4e8d926c Ntfs!NtfsCommonRead+0x1eb0
ffffa90d`f0b16ab0 fffff802`4c42a6b5 Ntfs!NtfsFsdRead+0x1fc
ffffa90d`f0b16b80 fffff802`485a70cf nt!IofCallDriver+0x55
ffffa90d`f0b16bc0 fffff802`485a4a03 FLTMGR!FltpLegacyProcessingAfterPreCallbacksCompleted+0x28f
ffffa90d`f0b16c30 fffff802`4c42a6b5 FLTMGR!FltpDispatch+0xa3
ffffa90d`f0b16c90 fffff802`4c4e4b27 nt!IofCallDriver+0x55
ffffa90d`f0b16cd0 fffff802`4c4f1d4a nt!IoPageReadEx+0x1d7
ffffa90d`f0b16d40 fffff802`4c4ef3cd nt!MiIssueHardFaultIo+0xb6
ffffa90d`f0b16d90 fffff802`4c439a48 nt!MiIssueHardFault+0x29d
ffffa90d`f0b16ea0 fffff802`4c60715e nt!MmAccessFault+0x468
ffffa90d`f0b17040 fffff802`4c47932b nt!KiPageFault+0x35e (TrapFrame @ ffffa90d`f0b17040)
ffffa90d`f0b171d0 fffff802`4c47889e nt!RtlpLookupFunctionEntryForStackWalks+0xcb
ffffa90d`f0b17250 fffff802`4c47851c nt!RtlpWalkFrameChain+0x34e
ffffa90d`f0b17960 fffff802`4c84f18d nt!RtlWalkFrameChain+0x11c
ffffa90d`f0b179b0 fffff802`4c84f530 nt!PoDiagCaptureUsermodeStack+0x45
ffffa90d`f0b179e0 fffff802`4c60a9b5 nt!NtSetTimerResolution+0x1a0
ffffa90d`f0b17a40 00007ffb`142106b4 nt!KiSystemServiceCopyEnd+0x25 (TrapFrame @ ffffa90d`f0b17a40)
0000002a`d4daf458 00000000`00000000 0x00007ffb`142106b4
Threads Waiting On Exclusive Access:
ffffaa094eed10c0 ffffaa094be65080 ffffaa093b47a040 ffffaa094b76d600
ffffaa094c440040 ffffaa094b781040 ffffaa094b678040 ffffaa094c6f7080
ffffaa094b80d080 ffffaa094b815040 ffffaa094d79a080 ffffaa094c3e4080
ffffaa094d68f080 ffffaa094d79c080
The thread incurred a hard page fault while holding the lock and has now held the lock for at least 588.5 seconds while it waits for the page I/O to complete; in the meantime, 14 other threads have begun to wait to acquire it.
The IRP looks like this:
Irp is active with 12 stacks 5 is current (= 0xffffaa094dd9c760)
Mdl=ffffaa094e5d25b0: No System Buffer: Thread ffffaa094e32f580: Irp stack trace.
cmd flg cl Device File Completion-Context
[N/A(0), N/A(0)]
0 0 00000000 00000000 00000000-00000000
Args: 00000000 00000000 00000000 00000000
[N/A(0), N/A(0)]
0 0 00000000 00000000 00000000-00000000
Args: 00000000 00000000 00000000 00000000
[N/A(0), N/A(0)]
0 0 00000000 00000000 00000000-00000000
Args: 00000000 00000000 00000000 00000000
[N/A(0), N/A(0)]
0 0 00000000 00000000 00000000-00000000
Args: 00000000 00000000 00000000 00000000
>[IRP_MJ_READ(3), N/A(34)]
10 e0 ffffaa093f4da0a0 00000000 fffff8024e4a2ff0-00000000 Success Error Cancel
\Driver\disk partmgr!PmIoCompletion
Args: 00004000 00000000 2895d39b400 00000000
[IRP_MJ_READ(3), N/A(3)]
10 e0 ffffaa093f3a78d0 00000000 fffff8024e4a3f10-00000000 Success Error Cancel
\Driver\partmgr partmgr!PartitionIoCompletion
Args: 4718a9b5 00000000 2895d39b400 00000000
[IRP_MJ_READ(3), N/A(0)]
10 e0 ffffaa093f43b060 00000000 fffff8024e591160-ffffaa093f3c0a40 Success Error Cancel
\Driver\partmgr volmgr!VmpReadWriteCompletionRoutine
Args: 00004000 00000000 955d29b400 00000000
[IRP_MJ_READ(3), N/A(0)]
0 e0 ffffaa093f3c08f0 00000000 fffff8024f1d52c0-ffffaa093f518180 Success Error Cancel
\Driver\volmgr fvevol!FvePassThroughCompletionRdpLevel2
Args: 00004000 00000000 4718a9ac 00000000
[IRP_MJ_READ(3), N/A(0)]
0 e0 ffffaa093f518030 00000000 fffff8024f3a5200-00000000 Success Error Cancel
\Driver\fvevol iorate!IoRateReadWriteCompletion
Args: 00004000 00000000 955d29b400 00000000
[IRP_MJ_READ(3), N/A(0)]
0 e0 ffffaa093f3718d0 00000000 fffff8024e8f3790-ffffa90df0b16a10 Success Error Cancel
\Driver\iorate Ntfs!NtfsMasterIrpSyncCompletionRoutine
Args: 00004000 00000000 955d29b400 00000000
[IRP_MJ_READ(3), N/A(0)]
0 e0 ffffaa0940c0f030 ffffaa0950133c80 fffff802485a44e0-ffffaa094ef5b010 Success Error Cancel
\FileSystem\Ntfs FLTMGR!FltpPassThroughCompletion
Args: 00004000 00000000 08f88400 00000000
[IRP_MJ_READ(3), N/A(0)]
0 1 ffffaa0940c395a0 ffffaa0950133c80 00000000-00000000 pending
\FileSystem\FltMgr
Args: 00004000 00000000 08f88400 00000000
Identifying the target storage device with !storagekd.storclass:
Storage Class Devices
Usage Legend: B = Boot, P = Paging, D = Dump, H = Hiber, R = Removable
FDO # Device ID Usage UP DN FL
-------------------------------------------------------------------------------
ffffaa093f4da0a0 [1,2] 0 WDC WD42EJRX-89BFNY0 ? ? 0
ffffaa093f408080 [1,2] 1 Samsung SSD 980 1TB BPDH ? ? 1
kd> !storagekd.storclass ffffaa093f4da0a0 2
Storage class device ffffaa093f4da0a0 with extension at ffffaa093f4da1f0
Classpnp Internal Information at ffffaa093f40e040
Transfer Packet Engine:
Packet Status DL Irp Opcode Sector/ListId UL Irp
-------- ------ -------- ------ --------------- --------
ffffaa093f3e43d0 Free ffffaa0938fced20
ffffaa094cf023d0 Free ffffaa094bebb060
ffffaa094cf02710 Free ffffaa094c9c1a20
ffffaa094cf02a50 Free ffffaa094cd27a20
ffffaa094cf06cf0 Queued ffffaa094ba21b50 00 ffffaa094dce8010 \Epic Games\Launcher\Engine\Binaries\ThirdParty\CEF3\Win64\libcef.dll
ffffaa094cf07030 Free ffffaa094a5b0a20
ffffaa094cf086f0 Free ffffaa094d10ea20
ffffaa094cf08210 Queued ffffaa094d10dd00 00 ffffaa094dd9c570 \Epic Games\Launcher\Engine\Binaries\ThirdParty\CEF3\Win64\libcef.dll
ffffaa094cf076b0 Free ffffaa094b8e8a60
ffffaa094e286710 Free ffffaa094e0b6a70
ffffaa094e286090 Free ffffaa094e241a70
ffffaa094e288930 Free ffffaa094d0ab920
ffffaa094e287750 Queued ffffaa094d722dd0 00 ffffaa094ebcf010 \Steam\logs\steamui_system.txt
ffffaa094e287270 Free ffffaa094df7ad70
ffffaa094cdf73f0 Queued ffffaa094e07fa70 00 ffffaa094ebe28c0 \Steam\logs\connection_log.txt
ffffaa094cdf7a70 Queued ffffaa094c394de0 00 ffffaa094ee65a70 \$LogFile
ffffaa09408ac720 Queued ffffaa094d54fde0 00 ffffaa094e2a2a30 \Epic Games\Launcher\Portal\Binaries\Win64\EpicGamesLauncher.exe
The read request is in a queue, waiting to be sent to the storage device; the storage device is not used for boot, paging, dump or hibernation; as we can see (!fxdevice), it is in a lower-power (D3) state:
Device Object: 0xffffaa093b475050
DevNode: 0xffffaa093b33c780
UniqueId: "\_SB.PC00.SAT0.PRT5"
InstancePath: "SCSI\Disk&Ven_WDC&Prod_WD42EJRX-89BFNY0\4&35b87281&0&050000"
Device Power State: PowerDeviceD3
Component Count: 1
Component 0: Current:F0/Deepest:F0 - ACTIVE (RefCount = 8)
libcef.dll (The Chromium Embedded Framework (CEF)) is a very large DLL (more than 100 megabytes) and has a very large PE “Exception Directory”; it also imports the routines timePeriodBegin/timePeriodEnd (which call NtSetTimerResolution) and RegisterPowerSettingNotification (which enables the DLL to learn of events like hibernation). NtSetTimerResolution walks the stack and needs to read the Exception Directory to do this on the x64 architecture.
All of the above factors combine to cause the bug check. Killing all instances of EpicWebHelper.exe before entering hibernation would help to verify the correctness of the analysis. Moving the installation of the software that uses/includes EpicWebHelper to the system disk would be a medium term workaround.
There is no easy “good” fix for this problem, but it is also not easy to exploit the issue to create an unprivileged tool to crash Windows; the difficulties include:
• Patience is needed; the crash would happen 10 minutes after initiating hibernation.
• A “non-system” disk is required to hold some DLLs.
• The timing of the call to timePeriodBegin/timePeriodEnd has to be finely judged – after requests to the storage device have been paused but while the thread can still run.
• Access to the exception table needs to cause a hard page fault.
Gary
