Hello everyone, thank you for joining.
My name is Pierre Rattimerik, I'm a PhD candidate at Fusek at Fry University at Amsterdam
and I'm in the Security Researcher mainly specializing in UEFI PCI Express and Hypervise
Security.
My previous work involves Intel Thunderbolt Fuller Ability Research and this is my colleague
Sina.
Hello everyone, I'm Sina, I'm also a PhD candidate at Fry University at Amsterdam.
I usually spend my time on system programming, maintaining Hyperdvise and like Windows
internals and a couple of digital designs, you can see more of my works here at my
block.
So we're giving two talks today at Fusek.
The current one is an introduction to Hyperdvise where we go into details on what
it can offer you but also how it works on the DeHood.
This morning we gave a talk in the Security Track where we discussed a case study of
using Hyperdvise for malware reversing.
If you missed that one, no worries, that will be recording.
Yeah, so let's talk about Hyperdvise.
We are currently sitting in the virtualization track but we're talking about B-Buggers, so
what's up with that?
So actually we're going to talk about both Hyperdvise and DeHood Buggers.
So why would you want a Hyperdvise or a B-Bugger?
Well, basically a Hyperdvise is highly preferred.
It has complete system visibility and that means that it has visibility over nearly
all events that are occurring in the operating system.
At the same time, because it is operating at the Hyperdvise level, it is also virtually
completely transparent to both user and kernel space.
In this case all sources of advantages, one being stealthy.
This is of course very useful when the bucking software that behaves differently when running
in a virtualized environment or when being debugged.
Yeah, being a hypervisor means we also have access to a toolbox that offers features
that are normally not possible with traditional debuggers.
So introducing Hyperdvise, it's a hypervisor, assisted debugger, leverages virtualization
extensions offered by the Intel ISA, originally intended for virtualization, but we're using
them to implement a debugger.
And because we are operating at the Hyperdvise level, we operate also independently of
any operating system level debugging APIs.
This gives all sorts of advantages and we'll be talking about those in a more detail.
The first release was for Windows back in 2022, actively maintained since and we're currently
working on a UEFI based, always acknowledged a hypervisor agent, which means that soon we
will be able to support Linux, BSD, and basically any other operating system.
So yeah, let's talk about some debugging scenarios, like if you have to debug native code,
especially when you don't have access to the source code, this can be really difficult.
For example, debugging a device driver, you're debugging the device driver, or you're
actually looking for a device driver that is touching a certain area in the memory, and you
don't know which device driver it is.
Well, this is exactly one of the use cases that Hyperdvise is useful for.
So you try to find what device driver, or what user space program is writing into the
certain memory range, it's no problem, you can find that out.
When you have finally found what device driver is writing into that certain memory range,
you might want to reverse trace the stack all the way up to user space.
You know, maybe there's a user space companion app talking to the device driver.
This is also possible with Hyperdvgy.
Maybe you want to script all of this, right?
Turn all of these data points into events that triggers certain scripts and does something
interesting.
Or maybe you want to prevent user kernel space, touching a range of memory.
And what the list goes on and goes on, these are all basically use cases that are not
possible with a traditional but debugger, and we'll show you how this can be done with Hyperdvgy.
So Hyperdvgy to the rescue, but before we do that, we have to introduce some new terms.
So Hyperdvgy implements a concept, but we like to call event-driven debugging.
So everything that happens in Hyperdvgy is basically an event, and each event could trigger
one or more actions. So it could be the execution of some script, it could be the execution
of a piece of assembly code that you wrote beforehand, and you could simply trigger a breakpoint.
And the inputs for these events can be anything from system calls or returns from system calls to
EPT hooks. What that is, we'll dive into in a little bit. It can also be IO operations, and it
can be even be specific CPU instructions. Like you know, that a certain piece of software,
either user or kernel code, uses these instructions. You can actually trigger an event based
on the use of this only instruction. So there are event calling stages, like post, pre, and both.
You can also do what we call like to call event short circuiting, which basically means you
define a set of conditions under which the event must be triggered. And then you can simply
ignore the entire event. This is how you can ignore read or write actions to a certain
memory region, for example. You can also completely bypass events, which means that
a certain range of instructions will be skipped. And we implement all of these using PMXids, but we'll
talk about that. Yeah, and so let's dive a little bit deeper into how we implement it all of
this. No, we're going to talk about the techniques, the way that we implement these features
in HyperDVG and how they can help us to enhance or reverse engineering experience.
The first, like, feature that we offer here is by using a monitor trap flag. If you are familiar
with virtualization of Intel VTX, there is like a VMCS and on top of that there is an MTF flag.
We made like a function tracking mechanism here that you can use to go from the user mode to
kernel mode and directly back from the kernel mode to user mode by employing MTF. And once it's combined
with symbols server, let's say Microsoft symbol server or a private symbol server, PDB files,
you can just see the name of the functions, the call, three of everything that is called.
And HyperDVG is capable of running from user mode to kernel mode and coming back from the
kernel to the user. This is something that traditional devices are not capable of doing that.
So in another scenario, we are trying to combine what we introduced before about even calling
stages and even short circuiting. There are three EScripts, there are three HyperDVG EScripts.
All you can see here is from the language of HyperDVG, we call DSLank. It's a customized language,
which is so similar to WindyVG EScripting language, but with a little bit of difference. So
this is the way that we create an event in HyperDVG. As you can see, we are trying to bypass the
execution of certain instructions. So first we go for MSORI, which is basically the WR MSORI,
if you're familiar with the model specific registers in Intel. And here we set like in the if you see
the stages here, we defined two stages as a pre-estate and post-estate.
And because the first example is a pre-estate, so the WR MSORI instruction is not yet executed.
And as a result, we could change the values that the WR MSORI instruction wants to put inside
like once to run on a bare metal machine. So they are all executing before the emulation happens
in the system. Another example is exceptions, you know, there are plenty of like in a Hypervisor,
there are exceptions, there are introps, there are traps. All of them can be monitored and can be
emulated by HyperDVG. So in this example, we use a post-calling stage here. And as a result,
like 0XE is a page fault. And based on Intel processors, there is like a location where the
page fault happened in CR2, which is there. So because we are using the post-estate, we could see
where this page fault happens and see the address. Another example is about changing system
calls. So we have two examples here. For the first example is that we are trying to intercept
the SQL instruction. And for the SQL instruction, first we check whether the SQL number,
which is always located in both Linux and Windows, all located on the first EA 3G server,
we check it with some values. Let's say 0XE55. And if it was like if the SQL number is 55,
then we change one of the parameters. Like if assuming that it's a SQL, it's a fast-go-calling
convention, we change some bits in ECX register or first parameter. And as you can see,
there's also another example here that we call it as the previous stage. So we have the capability
to bypass the system call. What happens here is that HyperDVG before executing and before emulating
the SQL instruction, checks for some conditions. And if this condition is like if the condition
met, then this event is short-circuited, which means that the SQL instruction is never emulated
and never executed in the virtualized environment. And what we do here, we change the RE,
searches there to put some status here. So the application, the debug application,
things that like the SQL instruction is successfully executed. And the value in REX,
which probably refers to like access denied file, is not able to be accessed,
is returned from the system call. But in reality, there is no SQL and nothing is executed in the
system. So this is how you can use short-circuiting. And you know, like you can also combine short-circuiting
with all of the events in HyperDVG. Let's say for events that are related to WDRMSR,
RGMSR, CPUID, TSC, like RDSC and RDSCP or RDPMC, all of the events that are available in HyperDVG
could be short-circuited. And another interesting example here is that how we can ignore
certain memory rights. Here by implementing these views, the EPD, second-level page table,
implementation of Intel. And as you can see, we put like a hook inside a certain address.
Let's say it's a variable inside a user mode application, or it could be also in a kernel mode,
like driver. And we check for all of the modifications like any memory rights into this address,
or memory reads into these address. So if there are, because it's W here, it means that
we want to intercept and trigger some events for memory rights. And we check for this
stage. So if the memory is equal to something like let's say a specific value,
then we could easily change that value as if nothing happens. And like we completely change the
value of the memory. How it could be useful as soon that you have a program that has different
threads, different threads that are accessing a specific global variable. And you just want to
make sure that this specific global variable is never changed to a certain value. Using this
as script, you can short-steer kit, or you can see or modify or prevent any modification of
the target memory. So what's make these things possible here?
We extensively use the MVAC or mode-based execution controls, which are available on Intel
processors for VTX. We have a mechanism in which we like allocate and use three different
EPTPs, so different EPTP pointers. And one of these EPTPs are a normal EPTP, and the
other EPTP is like a user mode execution denied, EPTP, and the other one is like
kernel mode execution denied, EPTP. Regarding MVAC, MVCC is introduced in Intel processors,
starting from KBLAG processors, I think it's seventh generation. And whenever we reach to our
target process, we just like detect whether we want one of these user denied or kernel denied
EPTPs in VMCS. So what we have here is that we combine or detect mode changing detection,
mechanism with some other features of Intel, which is called move to CR3 exiting. The thing is,
whenever the CR3 register is changed, we are sure that some context switches happens in the process,
and based on that, if we are interested on that process, we will load one of these three EPTPs.
So if the process is interesting for us, then we load one EPTP that detects, for example, the execution
of user mode. How using this approach using MVAC makes us capable of freezing the execution
of applications, I assume that we are targeting a specific process, and we are loading EPTPs for
that specific process that doesn't let it execute any user mode code. So what will happen here
is that the operating systems context switches to this specific process, and we only understand it
by intercepting all of the VMAs that are related to move to CR3 exiting. And in the context
switch, we just load the EPTP that denies the execution of user mode code. So what happens is that
at some point windows or the operating system tries to execute some user mode codes, and what will
happen is that a processor creates some EPTP violation of VMAs, and in EPTP violation VMAs,
it will just basically ignore it, as if we don't let the application to run. And at some point
the clock intrap comes and changes the process to another process. So what happens is that what
happens here is that the operating system thinks that the user mode code is running normally,
but in reality we just prevent any execution of user mode code. And this is the way that we implement
the time-freeze debugging for the user mode applications in HyperDouge. Here for the conclusion,
like HyperDouge has two different mechanisms, both for the debugging user mode and kernel mode.
So you can just debug an entire operating system with it. And it also leverages the modern hardware
features that gives it a system wide visibility. So it provides some powerful features
that are designed for reverse engineering, and these features are simply not possible with regular
debuggers, traditional debuggers, so you need a hypervisor to have those reverse engineering techniques.
And of course HyperDouge is a free and open source project, so it's available for the community
to contribute and has and patches are always welcome. Yes, that's it.
This is the on the low level instrumentation that you just do. Do you have any higher level
utilities to provide a bit more for example, OS awareness? Like I remember looking into it,
open VMI previously. So you mean the difference is that HyperDouge and open VMI?
Yes, maybe you can put it in context. Yes. Yes. Yes. Yes. So the question is what higher level
features we offer for like seeing some like higher level operating system content?
We do have like a specific script engine that is written for HyperDouge. And this
script engine has a lot of like it adverse about some like operating system concepts like
thread ID process ID and it also has like it has access to call certain let's say functions
from the Windows kernel or it is it has completely a complete interaction with the operating system.
So it's the higher level of debugging. Yeah, I would like to add that you can also use the Microsoft
Symbol server or your own private symbols to give context to the coach you're looking at.
But currently only because yes. No. Yes. So we're working on a UEFI based agent. So basically that
means our hypervisor will run at boot time and that way we can support Linux and be as the
and all the other operating systems. But it's a work in progress. So yeah, if you like helping us out,
please do. And also maybe it's good to add that in HyperDouge like the way that we designed
this script engine is completely different from the Windows G and we published an academic paper
out of it. And if you see that academic paper, it offers like more than 1,000, it is more than
1,000 times faster than Windows G because of its design.
Another question? Yeah.
To expand on that. So when you say this is working progress, are you like working to have
second run, like kind of and use a specific operating system?
Question 3. So the question is, can you? Are you going to implement kind of and use a specific
and running for all of the different operating systems you mentioned, like how are you going to
use a similar and understand what is going to happen? Yes. So the question is, if we are going to
support and add the support for all of the operating systems. Am I right?
I'm sorry. I'm sorry. So yes, actually the thing is like there are there are 13 things in HyperDouge
that are not related to the platform. And there are something that are related to the platform.
For example, we are using for now, we are using Intel VTX. So we have to execute those
instructions that are related to the VTX or VTD. But the thing is, like there are also like
like we have also used some mechanisms that are only available on Windows. Let's say IRQLs.
But the thing is, all of them can be applied to other operating systems as well. Like for example,
for IRQL, there is nothing like this in Linux. But we are trying to implement those specific
features for other operating systems as well. Yes.
So do you have for instance something where I can say I want to figure out every open system
about a Linux, like you have this information about what the system for Linux is?
So the question is, if we have like information about what the open systems also are in Linux.
Right now, I mean, what we did for implementing system calls can be applied also in Linux.
Like we are all of the events in hardware, the VG are using hypervisor capabilities. So there is
no, like it doesn't relate to Windows. But I mean, the user should also know about the
way that the system calls are implemented in Linux. For example, assume that Windows uses
a specific calling convention. Let's say fast call. And in Linux, the user should also know
about the fast call and the difference between fast calling and in Linux and Windows and
trace those systems calls. No, no, right, right now, HyperDVG only understands Windows.
So you mentioned your implement at the time, like, defined or in the
socialization process of Linux. How stealthy is that? Because as far as I understand, like,
the time, like, you're not diversity in kernel space, right? So if a kernel space
malware would monitor CIS calls, would they be able to catch it?
So the question is, how still see is the time freezing debugging inside
like Windows? Because we are only time freezing user mode applications. You want to know
whether like a kernel malware could bypass this or not. The thing is, in HyperDVG, there are two
like modes of debugging. Whether you can debug the kernel, entirely the kernel, and pause everything,
like pause the operating system, and nothing changes in the operating system. And the other thing
is the user mode code is like the user mode debugging is a separate thing. And the thing is,
like, if there is a malware that could run into the kernel, then there are like some limitations
for those malware. Like, for example, in Windows there is a driver signature enforcement. So
they have to sign it their driver, and there are also other ways of, like, for example,
who can those functions that are designed to load certain drivers, so we can just catch them from there.
Yes? I mean, I think that's a convenient way to figure out if you're running an VM, right?
Like, see, if you're ID, like, second layer, every sensation is only, you can't really hide all of those, right?
Yeah, so the question is, we can hide all of those footprints or not of the hypervisor. The thing is,
or first talk, in this morning, in the security track was exactly about this thing,
we cannot guarantee 100% transparency, but we, like, raise the bar, a lot, like hyperdvg,
even though by its nature is more steals, because it simply doesn't use API that Windows API
that has designed for debugging, but at the same time, we also try to mitigate those footprints
that are for hyperdvg itself. So I let me quickly pull up a slide from this morning that
exactly answers your question one second. So this is the roadmap that we have for
mitigating all of the artifacts that hypervisors, you know, are cells, but also to top left, left
will hypervisor could reveal to malware. We have the two on the left, we have implemented and we're
working on the ones in the middle there about 75% done, and the remaining ones, yeah, there's
scheduled right there. Do we have time for more questions? Sorry. Sorry, can you speak up?
Yes, well, we don't currently don't have plans for that, but it is certainly possible to implement
the same on Arm and AMD, yes. Yeah, go ahead.
As a member, you know, we have some commands, like, extension commands for SMIs, and we kind of
could, like, monitor certain things for the SMI handlers, for example, if I don't know an attacker or
any other application or the windows itself tries to call and tries to trigger some SMIs,
by using IO port certainly, you can use the features in hyperdvg to detect that, and we are also
having some commands for triggering SMIs, like, you just trigger the SMIs by using Hyperdvg,
if you check the document, there is the certain commands for SMI handlers, but in general,
everything is inside the VTX, inside the ring minus two hypervisor, not in SMM.
So you also kind of, I can't write a hook that would catch something, and it's actually not inside
SMI code. No, no, I mean, this is not possible in interpositive sort, like, you have to have,
like, access to the UEFI framework that loads the SMI handler and your code should be in SMM,
so this is technically not possible.
This is actually part of our effort to port hyperdvg to UEFI. We are currently not sure yet,
but it's feasible, but since we will be running from UEFI, we might be able to actually
intercept anything that's happening in SMM. But again, like, running into UEFI's, like,
you need to have, like, some access to the SPI chip, because, like, the SMI handler is loaded before the UEFI application.
Is that QMU? I mean, what we could do for the QMU is that we could
use some of their commands, like, some of their codes for emulating certain instructions.
But here, like, what we do is that we execute, like, we run on bare metal, so we don't have the plan to.
Yeah, yeah, I mean, like, right now, hyperdvg is supported on the VMware Workstation's nested virtualization
environment, and KVM, like, nested virtualization environment. It's not, it doesn't support hyperv,
we did, like, we tried a lot, but unfortunately, we couldn't port it to the hyperv, yet, but
right now, I think KVM is supported. Yeah, so the overhead comparison, normal hypervisors,
I mean, this is, like, the way that we emulated is exactly the same as the way that you emulated
like, like, let's say, like, I mean, once we execute those instructions,
I think we didn't have any measurements for this, but I think it should be generally
with less overhead compared to, like, big hypervisors, like, KVM, because, or VM exit handler
is much, has a much shorter path, and it's like the code that we wrote for it is, of course,
shorter than what they wrote for KVM. But, I mean, in any case, if you are running hyperdvg
in a nested virtualization environment, let's say, on a VMware, then the overhead of hyperdvg
is also added to the overhead of the VMware Workstation nested virtualization, because Intel doesn't
support, officially doesn't support nested virtualization in all, they just emulate everything,
so there is no nested virtualization support.