GICINGDoc
From VCGR Wiki
If you are interested in working on developing G-ICING, this page is for you. The original documentation is available at Media:GIcingDocumentation.doc. Here is a wiki-ized version of this document of the current working version of this document.
G-ICING Documentation
Last updated: May 24, 2009 Chris Sosa
- Terms
Installable File System Kit (IFS Kit) – This is the kit that Andrew bought from Microsoft that contains sample drivers as well as the necessary files to build the kernel component of the filesystem. This is installed at C:\WINDDK\{version} by default. If you build the kernel driver, you MUST have this installed. Due to licensing issues, we cannot legally put this into our repository.
NulMRx – The name of the sample project that the kernel driver is built on. The kernel driver still maintains the use of this name in many aspects of the code, including the registry entries and the name of the files generated.
Java Native Interface (JNI) – The name of the method for communicating between C\C++ (i.e. native code) and Java code.
GenesisIFSServer – The name of the user forwarding service. This program handles communication between the kernel and the Java code.
CToGenesisCalls – A library that GenesisIFSServer uses to actually communicate to the Java module through JNI
- File Locations
Non Java Code Locations
C:\WINDDK\3790.1830 – the default install location of the IFSKit. VSProjects use this directory by default and must be changed it this directory is changed.
C:\WINDDK\3790.1830\src – the location of sample drivers if you choose to install them along with the IFS Kit.
C:\WINDDK\3790.1830\src\filesys\nulmrx – the location within the build directory that contains the files for the driver. When updating the driver, you must build from this directory (or a sub-directory within it if you only want to build that sub-directory).
$DEVELOPMENT_PATH\src\jni\GenesisIFSServer – The location of the user mode module that talks to both the kernel driver and through the Java module
$DEVELOPMENT_PATH\src\jni\CToGenesisCalls – The locations of the library that the above program uses to actually make JNI calls to the Java module. A DLL gets built that GenesisIFSServer uses when this is compiled.
Java Code Locations
org.morgan.util.launcher.JniLibraryLauncher – This is the JNI skeleton on the Java side that CToGenesisCalls calls. All methods in this class are static and its primary role besides being a JNI skeleton is to dynamically load all the required libraries for the Genesis II client. This class mirrors Launcher which is used by the grid shell. It contains constants that can be changed to use Genesis II or the redirection library
edu.virginia.vcgr.genii.client.jni.gIIlib - package that contains all the code that is called by the aforementioned library launcher. This is where we keep track of Genesis II “file handles” and make calls to the common Genesis II caching framework. This is mostly Mark’s work.
edu.virginia.vcgr.genii.client.jni.giilibmirror – package that contains code that is the redirection library (see thesis). In short, this library mirrors filesystem calls back to the local harddrive (set in JNILibraryBase in the package – default C:\TestRoot). This is useful for testing the driver without Genesis II.
edu.virginia.vcgr.genii.client.gfs – the common Genesis II caching framework used by both the Linux Fuse filesystem and G-ICING.
Locations of built files
Once the code is built, it is important to note where the code is being built. The installer deals with placing this built code from their built locations into the correct locations for operation. However, I detail what the installer does in a latter part of this document.
$DEVELOPMENT_PATH\jni-lib\IFS – location of the most recently built IFS driver, user space executable, and associated files (nulmrx.sys, nulmrx.dll, GenesisServer.exe, nulmrx.ini, editRegistry.bat)
$DEVELOPMENT_PATH\jni-lib\CtoGenesisCalls.dll – location of the most recently built CToGenesisCalls.dll
Locations of where files need to be to be run
This lists the actual locations of where items need to be to be in the current “working” system.
$GENESIS_PATH\GenesisServer.exe – the path to the UFS executable
$GENESIS_PATH\lib – the path to the current java libraries
$GENESIS_PATH\jni-lib\CToGenesisCalls.dll – the path the CToGenesisCalls.dll must be in for the UFS to correctly use
\Windows\System32\nulmrx.dll – the path for nulmrx.dll (this should never really need to be updated)
\Windows\System32\Drivers\nulmrx.sys – the path for the kernel driver. If changed, the machine must be restarted to use the new driver
- Installation Procedure
There are two methods for installing the driver: the automatic method that is bundles with the Genesis II installer, or the manual method. The former is more useful for individuals interested in testing / using the driver while the latter is more important for those debugging the driver.
Automatic Installation
The Genesis II installer comes bundled with G-ICING. To install, select install IFS. This option is only available if you are installing in a Windows OS. In summary, this edits the registry, copies the driver files into C:\Windows\System32 correctly and copies GenesisServer.exe up to the installation directory. Once you install a new version of the G-ICING driver, you will need to restart your computer before you’ll be able to use it.
Manual Installation
As mentioned, the installation consists of three parts:
1) Copying the nulmrx.sys, and nulmrx.dll into Windows\System32\Drivers and Windows\System32 appropriately
2) Copying GenesisServer.exe to the installation directory.
3) Editing the registry by using the nulmrx.ini (in jni-lib\IFS). There is a batch file at jni-lib\IFS\editRegistry.bat that does this automatically.
If you develop and modify GenesisServer.exe, or the kernel driver, you’ll have to recopy these to the appropriate directories.
- Kernel Management Service
Overview of the interesting files
NulMrxExtension.sln – the Visual Studio 2005 solution for the kernel service. This lets you view the code but does not let you build the code.
init.c – contains initialization code including initializing the function tables (the entry points to the driver), device code to handle request and responses from the UFS in the inverted call model and other miscellaneous functions. The most important code that might need to be changed is under ProcessResponse and ProcessControlRequest. These correspond to when the UFS sends a new request (the first call of the inverted call model) and a response (the second part).
nulmrx.h – where the code common to both the KMS and UFS is stored. This contains constants for the inverted call model, device names and common structures. If changes to the KMS / UFS interface need to be done, here is one area that would be of interest.
mrxglobs.h – code common to all the KMS but not part of the interface. This includes structures for both devices, specific Genesis file control blocks, context blocks and other miscellaneous structures. Also, all functions that are shared across modules are declared here.
invertedcall.c – code that deals with the majority of sending a request to the UFS from the KMS. This includes code to prepare statements for different operations. This code is pretty much straight forward modifications from the sample provided by OSROnline. Check that tutorial for more information.
openclos.c – code that deals with opens and closes. In Windows opens (creates) do everything … they create files/directories as well as open them. Close is a complicated procedure. Please see the IFS Kit documentation for more information about closes.
read.c, write.c, dirctrl, fileinfo.c – handles reads, writes, directory calls, and information-related calls respectively. By information-related calls I mean calls that are similar to stat and fstat.
downvli.c – actually handles renames and extends of files. In Windows, files must be extended before they can be written to i.e. you will always get an extend from 5024 -> 6098 before the filesystem will request to write to byte 5025.
The rest of the .c files are in their original form from the sample or have been changed in small ways to remove some printf’s (DbgPrint in Kernel land).
Building the kernel driver
First and foremost, to build the driver, all the kernel code must be located within the IFS Kit directory. Specifically it needs to be under
C:\WINDDK\3790.1830\src\filesys\nulmrx\wxp\sys
Therefore, you must have the IFS kit installed in order to build this driver. Once you have the code in the correct location, select the checked build under Windows XP in the IFS Kit start menu item and navigate to the aforementioned directory and type:
build –cZg
This builds the current directory (and any nested source directories) and places the generated objects in the object folder. The new nulmrx.sys will be placed here.
- The User Forwarding Service
Overview of the interesting files
GenesisIFSServer.sln – the Visual Studio 2005 solution for building and viewing the UFS module.
Server.c – the main (only) module for the UFS. This server spawns pre-defined (by a constant) number of threads that send requests to the KMS awaiting requests from the Inverted Call Model. Much of this code has been taken from the OSROnline Inverted Call Model sample. This project also links to the CToGenesisCalls.dll that must have been built before. This is used to make JNI calls (calls that are prefixed with “genesisII_”)
Building the UFS module
Just compile using Visual Studio. You may have to check the paths in the project properties if you have unusual installation locations.
- CToGenesisCalls library
Overview of interesting files
CToGenesisCalls.sln – The Visual Studio 2005 solution for viewing and building this library
MakeGenesisCalls.h – the header file that documents and declares the interface to the library i.e. there are the functions that the GenesisServer can use.
MakeGenesisCalls.c – the main file of the project. This includes all the functions for calling into the Genesis II java module. Look at JNI documentation for more information about constants and types used here.
Building the Library
Just build using Visual Studio. This might be tricky because the link must be able to find the jvm.dll for your Java installation. This should be in your JRE directory\bin\client. You may have to edit the project properties to enter the path of your JRE directory if it is different than my default.
- G-ICING Java module
I have elected not to document much of this code except the entry point as it is fairly well documented and besides the below classes, are much more Mark’s code than my own. One should be able to get around in this code knowing the locations of the libraries (section 2) and with the help of Mark.
Important classes
org.morgan.util.launcher.JNILibraryLauncher – the JNI stubs on the Java end. These are all static methods that are called by the CToGenesisCalls library and the entry point to the Java code. This class is also responsible for dynamically loading the other libraries of Genesis II.
edu.virginia.vcgr.genii.client.jni.gIIlib.JNILibraryBase - class contains a lot of constants used by the project.
To build
The JNILibraryLauncher is built along with the morgan-utilties.jar while the rest of the code is built with the GenesisIIClient.jar. You can use ant to build both of these jar files.
- Some notes of Kernel Debugging
In order to debug/develop the kernel driver, you will need two machines and a null serial cable. If you do not have this available, use a Virtual Machine to emulate the second machine. There is a fair amount of information out there in how to set up these machines. I personally used MS Virtual PC 2007 with great success. Please see my thesis for more information regarding this choice. Since a lot of this information is available out there I will be brief.
First, there are two machines to keep in mind. The host machine (the control machine where you are probably doing your debugging and probably your main machine for other tasks), and the target machine. With a VM, the VM hosts the target machine. In the newly created machine, you must enable kernel debugging to be forwarded. To do this, you must edit the boot file. Again, for whatever setup you want, it is best to look this up.
Once you have are forwarding debugging information, you have to have an actual debugger collect this information on the host machine. I used (and it is recommended you use) WinDbg. This is free and provided by MS. You should have this on with the correct settings to connect to the target machine (this again depends on your configuration). Once this connection is set up, you can start WinDbg and then start the VM. Once this is done, you will start seeing DbgPrintf’s from your kernel driver when it is loaded and whenever else it makes these calls.
Besides print statements, you can also break into the kernel code. To do this, you must statically enter breakpoints by using DbgBreakPnt(line number). When the kernel executes this line, it breaks into the debugger as a normal debugger would.
On the case of a bug check (a BSOD), the target machine will send a lot of debugging information to WinDbg. To make sense of this, type analyze -v. This will give you a stack trace of where the bug check occurred as well as a guess as to what caused it.
- Current Setup on Trajan
Before leaving, I left Trajan set up with an environment similar to what I had. This section details specifics about this setup and how to use it.
Working with Virtual Machines
For my development environment, I chose to use Virtual Machines as my target machines (see section 8 for what I mean by “target”). Specifically I set up Microsoft Virtual PC virtual machines. These machines and their corresponding hard drive files are located at:
C:\Virtual Machines\Virtual Machine Disks
You’ll notice if you navigate to C:\Virtual Machines, there are two other sub folders that will be of interest:
C:\Virtual Machines\Shared Drive – To copy files between the host machine and the target machine, the target machine has mounted a folder from the host machine as its S:\ drive. This is that shared drive. Therefore anything you place in this folder will appear in the S:\ drive in the virtual machines.
C:\Virtual Machines\Scripts – This is the location of scripts that are shortcuts to copy development copies of your programs to the Virtual Machine. I will go into this a bit further in the section 9.3.
To start a virtual machine, navigate to Microsoft Virtual PC. If this is your first time, you will have to add one of the existing virtual machines to Virtual PC. To do this, select open existing in Virtual PC’s menu and navigate to the C:\Virtual Machines\Virtual Machine Disks. Choose either VM for your use here.
Once Virtual PC recognizes one of these VM’s, you can start by double clicking on it or highlighting it and selecting start VM. You can also change the properties of the VM by selecting “settings” in the menu. You can items such as the size of main memory from there.
You should do some reading on “Undo Disks”. I found them quite useful and I advise that you leave them on.
Once either VM starts to boot, a window appears showing the machine booting up. You’re given an option of which OS to run. Both are XP, however, one mentions debugging enabled. If you would like to see print statements or breakpoints appear in your debugger, you need to have debugging enabled.
Debugging
In order to debug a VM, you must use WinDbg. This is installed on Trajan and can be found in the Windows start menu. I have already set it up, however, if you don’t start seeing debug information once the OS starts to boot up, then you need to go into WinDbg File->Open Workspace and select Default. I have set this up for your use but it might not load automatically if you are not me. That’s it. If you set breakpoints using DbgBreakPoint(line number) or DbgPrintLn’s, you will either break into the debugger or you’ll receive the println’s on the console from the driver.
There’s not much else to it in terms of the setup. See section 8 for more of an overview about debugging.
Scripts
There are three scripts in the scripting directory:
CopyGenesisLib.bat – This copies the Genesis II client code from the working Eclipse directory (set in the first line of the .bat script) to the Virtual Machine’s shared drive. Use this if you’ve made modifications to the Java code and need to update the changes to the test environment.
CopyToShare.bat – This copies the C code (the kernel driver, the GenesisServer and the .dll) over to the Shared Drive.
CopyToSVN.bat – As I advise developing the kernel driver within the WinDDK directory, you need to at some point update your changes to the SVN repository. This script copies all your changes over to the eclipse directory. Next time you commit changes to the project, these changes will be updated.
I would recommend putting shortcuts to these batch scripts on your desktop.
Also, there are two relevant scripts on the VM’s desktop that you’ll need to use in conjunction with the first two scripts mentioned above.
CopyGeniiLib.bat – The partner of CopyGenesisLib.bat. This copies updated Java libraries from the shared drive to the working Genesis directory in the VM.
InstallDriver.bat – Copies the C code to the correct locations in the VM. If you’ve modified the kernel driver, you will need to restart the VM to see these changes.
