Inspect pages created after a vm_object is marked as copy-on-write

Mihai Carabas mihai.carabas at
Sat Jun 30 07:38:25 UTC 2018

On Sat, Jun 30, 2018 at 1:52 AM, Mark Johnston <markj at> wrote:
> On Fri, Jun 29, 2018 at 11:58:31AM +0300, Elena Mihailescu wrote:
>>  Hello,
>> I am interested if there is a method to inspect what pages/objects were
>> created after a vm_object (the vm_map_entry associated with the object) is
>> marked as copy-on-write. More specifically, I'm interested only in the
>> pages that were copied when a write operation was proceed on a page that
>> belongs to the object marked copy-on-write.
>> I need this for a live migration feature for bhyve in order to send the
>> pages that were modified between the iterations in which I migrate the
>> guest's memory(the guest's memory will be migrated in rounds - firstly, all
>> memory will be sent remote, then, only the pages that were modified and so
>> on).
>> What I want to implement is the following:
>> Step 1: Given a vm_object *obj, mark its associated vm_map_entry *entry as
>> copy-on-write.
>> Step 2: After a while (a non-deterministic amount of time),
>> inspect/retrieve the pages that were created based on information existent
>> in the object.
>> What I tried until now:
>> I implemented a function in kernel that:
>> - gets the vmspace structure pointer for the current process
>> - gets the vm_map structure pointer for the vmspace
>> - iterates through each vm_map_entry and based on the vm_offset_start and
>> vm_offset_end determines vm_map_entry that contains the object I am
>> interested in.
>> - for this object, it prints some debug information such as: shadow_count,
>> ref_count, whether if it has a backing_object or not.
>> The code written is similar with the code from here (the way in which I get
>> vmspace for the current process and the way I am iterating through
>> vm_map_entry and objects):
>> [0]
>> I have read the following documentation about FreeBSD's implementation for
>> virtual memory:
>> [1]
>> [2]
>> [3]
>> [4]
>> As far as I could tell after reading the documentation presented above, I
>> should look for the object that the object I am interested in is a shadow
>> of or an object that my object is shadow for.
> Right. When a copy-on-write fault results in the creation of a new
> object, the new object is said to shadow the original object, which
> becomes the backing object for the shadow object.  When faults in the
> corresponding map entry occur, the fault handler first searches for a
> page in the map entry's object, and then falls back to the backing
> object if necessary.
>> To do that, I should inspect the following fields from the vm_object
>> structure (among others)(
>> :
>> - int shadow_count; /* how many objects that this is a shadow for */
>> - struct vm_object *backing_object; /* object that I'm a shadow of */
>> But in all my tests, for the object I am interested in, the shadow_count is
>> 0 and the backing_object is NULL.
>> The code I use to mark the vm_map_entry for the object I am interested in
>> copy-on-write is here:
>> [5]
> MAP_ENTRY_NEEDS_COPY needs to be set in order for the copy-on-write
> machinery to work the way you expect.  Take a look at vm_map_lookup():
> when it sees that flag and the caller is attempting a write fault, it
> creates a shadow object, updates the entry and clears
>> Is there anything I am doing wrong? Maybe I misunderstood something about
>> the way the virtual memory works in FreeBSD.
> I'll note that inspecting and manipulating vm_map_entry and vm_object
> structures in the bhyve code constitutes something of an abstraction
> violation, though it's reasonable to proceed this way while working on a
> prototype of the feature.  That is, I think you should keep trying your
> current approach, but just be aware that you are using the copy-on-write
> mechanism in a way that the VM system isn't really expecting.

Can you point out the right approach in our case?


>> There is another way I could inspect what pages were created between the
>> moment I mark an object (its vm_map_entry) as copy-on-write and a later
>> moment?
> _______________________________________________
> freebsd-virtualization at mailing list
> To unsubscribe, send any mail to "freebsd-virtualization-unsubscribe at"

More information about the freebsd-amd64 mailing list