Hello, dear friend, you can consult us at any time if you have any questions, add WeChat: THEend8_
Introduction
In this lab, you will implement process memory isolation, virtual memory, and a system call (fork()) in a tiny
(but real!) operating system, called WeensyOS.
This will introduce you to virtual memory and reinforce some of the concepts that we have covered this
semester.
The WeensyOS kernel runs on x86-64 CPUs. Because the OS kernel runs on the “bare” hardware, debugging
kernel code can be tough: if a bug causes misconfiguration of the hardware, the usual result is a crash of the
entire kernel (and all the applications running on top of it). And because the kernel itself provides the most
basic system services (for example, causing the display hardware to display error messages), deducing what
led to a kernel crash can be particularly challenging. In the old days, the usual way to develop code for an OS
(whether as part of a class, in a research lab, or in industry) was to boot it on a physical CPU. The lives of
kernel developers have gotten much better since. You will run WeensyOS in QEMU.
QEMU is a software-based x86-64 emulator: it “looks” to WeensyOS just like a physical x86-64 CPU with a
particular hardware configuration. However, if your WeensyOS code-in-progress wedges the (virtual)
hardware, QEMU itself and the whole OS that is running on the “real” hardware (that is, the “real” Linux OS that
QEMU is running on) survive unscathed (“real” is in quotation marks for reasons that will be unpacked in the
next paragraph). So, for example, your last few debugging printf()s before a kernel crash will still get logged
to disk (by QEMU running on Linux), and “rebooting” the kernel you?re developing amounts to re-running the
QEMU emulator application.
What is the actual software/hardware stack here? The answer is different for students with x86-64 computers
(for example, Windows machines and older Macs) and ARMs. All students are running a host OS (on your
computer) on top of either x86-64 or ARM hardware (ARM being the architecture for so-called Apple silicon,
namely M1 and M2 chips). Then, the Docker containerization environment runs on top of the host OS (as a
process). That environment, loosely speaking, emulates either an x86 or an ARM CPU, and running on top of
that emulated CPU is Ubuntu Linux, targeted to x86-64 or ARM. Running on top of Ubuntu is QEMU. QEMU
presents an emulated x86-64 interface, and QEMU itself is either an x86-64 or ARM binary, again depending
on the underlying hardware. Finally, WeensyOS is exclusively an x86-64 binary, and that of course runs on
QEMU (though if you have some x86-64 hardware sitting around, you can try installing WeensyOS and
running it “bare”). Taking that same progression, now top-down: if you have an ARM CPU, that means you are
running the WeensyOS kernel?s x86-64 instructions in QEMU, a software-emulated x86-64 CPU that is an
ARM binary, on top of Linux (targeted to ARM), running in the Docker containerization environment (also itself
an ARM binary), on macOS, running on an ARM hardware CPU.
Heads up. As always, it?s important to start on time. In this case, on time means 3 weeks before the
assignment is due, as you will almost certainly need all of the allotted time to complete the lab. Kernel
development is less forgiving than developing user-level applications; tiny deviations in the configuration of
hardware (such as the MMU) by the OS tend to bring the whole (emulated) machine to a halt.
To save yourself headaches later, read this lab writeup in its entirety before you begin.
Resources.
4/2/24, 1:01 AM CS202: Lab 4: WeensyOS
https://cs.nyu.edu/~mwalfish/classes/24sp/labs/lab4.html 2/19
You may want to look at Chapter 9 of CSAPP3e (from which our x86-64 virtual memory handout is
borrowed). The book is on reserve at the Courant library. Section 9.7 in particular describes the 64-bit
virtual memory architecture of the x86-64 CPU. Figure 9.23 and Section 9.7.1 show and discuss the
PTE_P, PTE_W, and PTE_U bits; these are flags in the x86-64 hardware?s page table entries that play a
central role in this lab.
You may find yourself during the lab wanting to understand particular assembly instructions. Here are
two guides to x86-64 instructions, from Brown and CMU. The former is more digestible; the latter is
more comprehensive. The supplied code also uses certain assembly instructions like iret; see here
for a reference.
Getting Started
You?ll be working in the Docker container as usual. We assume that you have set up the upstream as described
in the lab setup. Then run the following on your local machine (Mac users can do this on their local machine or
within the Docker container; Windows and CIMS users should do this from outside the container):
$ cd ~/cs202
$ git fetch upstream
$ git merge upstream/main
This lab?s files are located in the lab4 subdirectory.
If you have any “conflicts” from lab 3, resolve them before continuing further. Run git push to save your work
back to your personal repository.
Another heads up. Given the complexity of this lab, and the possibility of breaking the functionality of the
kernel if you code in some errors, make sure to commit and push your code often! It's very important that
your commits have working versions of the code, so if something goes wrong, you can always go back to a
previous commit and get back a working copy! At the very least, for this lab, you should be committing
once per step (and probably more often), so you can go back to the last step if necessary.
Goal
You will implement complete and correct memory isolation for WeensyOS processes. Then you'll implement
full virtual memory, which will improve utilization. You'll implement fork() (creating new processes at runtime)
and for extra credit, you?ll implement exit() (destroying processes at runtime).
We?ve provided you with a lot of support code for this assignment; the code you will need to write is in fact
limited in extent. Our complete solution (for all 5 stages) consists of well under 300 lines of code beyond what
we initially hand out to you. All the code you write will go in kernel.c (except for part of step 6).
Testing, checking, and validation
For this assignment, your primary checking method will be to run your instance of Weensy OS and visually
compare it to the images you see below in the assignment.
Studying these graphical memory maps carefully is the best way to determine whether your WeensyOS code
for each stage is working correctly. Therefore, you will definitely want to make sure you understand how to
read these maps before you startto code.
4/2/24, 1:01 AM CS202: Lab 4: WeensyOS
https://cs.nyu.edu/~mwalfish/classes/24sp/labs/lab4.html 3/19
We supply some grading scripts, outlined at the end of the lab, but those will not be your principal source of
feedback. For the most part, they indicate only whether a given step is passing or failing; look to the memory
maps to understand why.
Initial state
Enter the Docker environment:
$ ./cs202-run-docker
cs202-user@172b6e333e91:~/cs202-labs$ cd lab4/
cs202-user@172b6e333e91:~/cs202-labs/lab4$ make run
The rest of these instructions presume that you are in the Docker environment. We omit the cs202-
user@172b6e333e91:~/cs202-labs part of the prompt.
make run should cause you to see something like the below, which shows four processes running in parallel,
each running a version of the program in p-allocator:
This image loops forever; in an actual run, the bars will move to the right and stay there. Don't worry if your
image has different numbers of K's or otherwise has different details.
If your bars run painfully slowly, edit the p-allocator.c file and reduce the ALLOC_SLOWDOWN constant.
Stop now to read and understand p-allocator.c.
Here?s how to interpret the memory map display:
WeensyOS displays the current state of physical and virtual memory. Each character represents 4 KB
of memory: a single page. There are 2 MB of physical memory in total. (Ask yourself: how many pages
is this?)
WeensyOS runs four processes, 1 through 4. Each process is compiled from the same source code (pallocator.c), but linked to use a different region of memory.
Each process asks the kernel for more heap memory, one page at a time, until it runs out of room. As
usual, each process's heap begins just above its code and global data, and ends just below its stack.