| <chapter> |
| <title>Kernel modules</title> |
| <para> |
| This section covers the kernel modules. As already stated, Wine |
| implements the NT architecture, hence provides <filename>NTDLL</filename> |
| for the core kernel functions, and <filename>KERNEL32</filename>, which is |
| the implementation of the basis of the Win32 subsystem, on top of |
| <filename>NTDLL</filename>. |
| </para> |
| <para> |
| This chapter is made of two types of material (depending of their point of |
| view). Some items will be tackled from a global point of view and then, |
| when needed, explaining the split of work between |
| <filename>NTDLL</filename> and <filename>KERNEL32</filename>; some others |
| will be tackled from a DLL point of view (<filename>NTDLL</filename> or |
| <filename>KERNEL32</filename>). The choice is made so that the output is |
| more readable and understantable. At least, that's the intend (sigh). |
| </para> |
| |
| <sect1 id="initialization"> |
| |
| <title>The Wine initialization process</title> |
| |
| <para> |
| Wine has a rather complex startup procedure, so unlike many programs the |
| best place to begin exploring the code-base is |
| <emphasis>not</emphasis> in fact at the <function>main()</function> |
| function but instead at some of the more straightforward DLLs that |
| exist on the periphery such as MSI, the widget library (in |
| <filename>USER</filename> and <filename>COMCTL32</filename>) etc. The |
| purpose of this section is to document and explain how Wine starts up |
| from the moment the user runs "<command>wine myprogram.exe</command>" to |
| the point at which <filename>myprogram</filename> gets control. |
| </para> |
| |
| <sect2> |
| <title>First Steps</title> |
| |
| <para> |
| The actual wine binary that the user runs does not do very much, in |
| fact it is only responsible for checking the threading model in use |
| (NPTL vs LinuxThreads) and then invoking a new binary which performs |
| the next stage in the startup sequence. See the beginning of this |
| chapter for more information on this check and why it's necessary. You |
| can find this code in <filename>loader/glibc.c</filename>. The result |
| of this check is an exec of either <command>wine-pthread</command> or |
| <command>wine-kthread</command>, potentially (on Linux) via the |
| <emphasis>preloader</emphasis>. We need to use separate binaries here |
| because overriding the native pthreads library requires us to exploit |
| a property of ELF symbol fixup semantics: it's not possible to do this |
| without starting a new process. |
| </para> |
| |
| <para> |
| The Wine preloader is found in |
| <filename>loader/preloader.c</filename>, and is required in order to |
| impose a Win32 style address space layout upon the newly created Win32 |
| process. The details of what this does is covered in the address space |
| layout chapter. The preloader is a statically linked ELF binary which |
| is passed the name of the actual Wine binary to run (either |
| <command>wine-kthread</command> or <command>wine-pthread</command>) |
| along with the arguments the user passed in from the command line. The |
| preloader is an unusual program: it does not have a |
| <function>main()</function> function. In standard ELF applications, |
| the entry point is actually at a symbol named |
| <function>_start()</function>: this is provided by the |
| standard <command>gcc</command> infrastructure and normally jumps to |
| <function>__libc_start_main()</function> which initializes glibc before |
| passing control to the main function as defined by the programmer. |
| </para> |
| |
| <para> |
| The preloader takes control direct from the entry point for a few |
| reasons. Firstly, it is required that glibc is not initialized twice: |
| the result of such behaviour is undefined and subject to change |
| without notice. Secondly, it's possible that as part of initializing |
| glibc, the address space layout could be changed - for instance, any |
| call to <function>malloc()</function> will initialize a heap arena |
| which modifies the VM mappings. Finally, glibc does not return to |
| <function>_start()</function> at any point, so by reusing it we avoid |
| the need to recreate the ELF bootstrap stack |
| (<varname>env</varname>, <varname>argv</varname>, auxiliary array etc). |
| </para> |
| |
| <para> |
| The preloader is responsible for two things: protecting important |
| regions of the address space so the dynamic linker does not map shared |
| libraries into them, and once that is done loading the real Wine |
| binary off disk, linking it and starting it up. Normally all this is |
| automatically by glibc and the kernel but as we intercepted this |
| process by using a static binary it's up to us to restart the |
| process. The bulk of the code in the preloader is about loading |
| <command>wine-[pk]thread</command> and |
| <filename>ld-linux.so.2</filename> off disk, linking them together, |
| then starting the dynamic linking process. |
| </para> |
| |
| <para> |
| One of the last things the preloader does before jumping into the |
| dynamic linker is scan the symbol table of the loaded Wine binary and |
| set the value of a global variable directly: this is a more efficient |
| way of passing information to the main Wine program than flattening |
| the data structures into an environment variable or command line |
| parameter then unpacking it on the other side, but it achieves pretty |
| much the same thing. The global variable set points to the preload |
| descriptor table, which contains the VMA regions protected by the |
| preloader. This allows Wine to unmap them once the dynamic linker has |
| been run, so leaving gaps we can initialize properly later on. |
| </para> |
| |
| </sect2> |
| |
| <sect2> |
| <title>Starting the emulator</title> |
| |
| <para> |
| The process of starting up the emulator itself is mostly one of |
| chaining through various initializer functions defined in the core |
| libraries and DLLs: <filename>libwine</filename>, then |
| <filename>NTDLL</filename>, then <filename>KERNEL32</filename>. |
| </para> |
| |
| <para> |
| Both the <command>wine-pthread</command> and |
| <command>wine-kthread</command> binaries share a common |
| <function>main()</function> function, defined in |
| <filename>loader/main.c</filename>, so no matter which binary is |
| selected after the preloader has run we start here. This passes the |
| information provided by the preloader into |
| <filename>libwine</filename> and then calls |
| <function>wine_init()</function>, defined in |
| <filename>libs/wine/loader.c</filename>. This is where the emulation |
| really starts: |
| <function>wine_init()</function> can, with the correct preparation, |
| be called from programs other than the wine loader itself. |
| </para> |
| |
| <para> |
| <function>wine_init()</function> does some very basic setup tasks such |
| as initializing the debugging infrastructure, yet more address space |
| manipulation (see the information on the 4G/4G VM split in the address |
| space chapter), before loading <filename>NTDLL</filename> - the core |
| of both Wine and the Windows NT series - and jumping to the |
| <function>__wine_process_init()</function> function defined |
| in <filename>dlls/ntdll/loader.c</filename> |
| </para> |
| |
| <para> |
| This function is responsible for initializing the primary Win32 |
| environment. In <function>thread_init()</function>, it sets up the |
| TEB, the <command>wineserver</command> connection for the main thread |
| and the process heap. See the beginning of this chapter for more |
| information on this. |
| |
| </para> |
| |
| <para> |
| Finally, it loads and jumps to |
| <function>__wine_kernel_init()</function> in |
| <filename>KERNEL32.DLL</filename>: this is defined in |
| <filename>dlls/kernel32/process.c</filename>. This is where the bulk |
| of the work is done. The <filename>KERNEL32</filename> initialization |
| code retrieves the startup info for the process from the server, |
| initializes the registry, sets up the drive mapping system and locale |
| data, then begins loading the requested application itself. Each |
| process has a <structname>STARTUPINFO</structname> block that can be |
| passed into <function>CreateProcess</function> specifying various |
| things like how the first window should be displayed: this is sent to |
| the new process via the <command>wineserver</command>. |
| </para> |
| |
| <para> |
| After determining the type of file given to Wine by the user (a Win32 |
| EXE file, a Win16 EXE, a Winelib app etc), the program is loaded into |
| memory (which may involve loading and initializing other DLLs, the |
| bulk of Wines startup code), before control reaches the end of |
| <function>__wine_kernel_init()</function>. This function ends with the |
| new process stack being initialized, and start_process being called on |
| the new stack. Nearly there! |
| </para> |
| |
| <para> |
| The final element of initializing Wine is starting the newly loaded |
| program itself. <function>start_process()</function> sets up the SEH |
| backstop handler, calls <function>LdrInitializeThunk()</function> |
| which performs the last part of the process initialization (such as |
| performing relocations and calling the <function>DllMain()</function> |
| with <constant>PROCESS_ATTACH</constant>), grabs the entry point of |
| the executable and then on this line: |
| </para> |
| |
| <programlisting> |
| ExitProcess( entry( peb ) ); |
| </programlisting> |
| |
| <para> |
| ... jumps to the entry point of the program. At this point the users |
| program is running and the API provided by Wine is ready to be |
| used. When entry returns, the <function>ExitProcess()</function> API |
| will be used to initialize a graceful shutdown. |
| </para> |
| </sect2> |
| </sect1> |
| |
| <sect1> |
| <title>Detailed memory management</title> |
| <para> |
| As already explained in previous chapter (see <xref linkend="arch-mem"> |
| for the details), Wine creates every 32-bit Windows process in its own |
| 32 address space. Wine also tries to map at the relevant addresses what |
| Windows would do. There are however a few nasty bits to look at. |
| </para> |
| |
| <sect2> |
| <title>Implementation</title> |
| <para> |
| Wine (with a bit of black magic) is able to map the main module at |
| it's desired address (likely <constant>0x400000</constant>), to create |
| the process heap, its stack (as a Windows executable can ask for a |
| specific stack size), Wine simply use the initial stack of the ELF |
| executable for its initialisation, but creates a new stack (as a Win32 |
| one) for the main thread of the executable. Wine also tries to map all |
| native DLLs at their desired address, so that no relocation has to be |
| performed. |
| </para> |
| <para> |
| Wine also implements the shared heap so native win9x DLLs can be |
| used. This heap is always created at the |
| <constant>SYSTEM_HEAP_BASE</constant> address or |
| <constant>0x80000000</constant> and defaults to 16 megabytes in size. |
| </para> |
| <para> |
| There are a few other magic locations. The bottom 64k of memory is |
| deliberately left unmapped to catch null pointer dereferences. The |
| region from 64k to 1mb+64k are reserved for DOS compatibility and |
| contain various DOS data structures. Finally, the address space also |
| contains mappings for the Wine binary itself, any native libaries |
| Wine is using, the glibc malloc arena and so on. |
| </para> |
| </sect2> |
| |
| <sect2 id="address-space"> |
| <title>Laying out the address space</title> |
| |
| <para> |
| Up until about the start of 2004, the Linux address space very much |
| resembled the Windows 9x layout: the kernel sat in the top gigabyte, |
| the bottom pages were unmapped to catch null pointer dereferences, and |
| the rest was free. The kernels mmap algorithm was predictable: it |
| would start by mapping files at low addresses and work up from there. |
| </para> |
| |
| <para> |
| The development of a series of new low level patches violated many of |
| these assumptions, and resulted in Wine needing to force the Win32 |
| address space layout upon the system. This section looks at why and |
| how this is done. |
| </para> |
| |
| <para> |
| The exec-shield patch increases security by randomizing the kernels |
| mmap algorithms. Rather than consistently choosing the same addresses |
| given the same sequence of requests, the kernel will now choose |
| randomized addresses. Because the Linux dynamic linker |
| (<filename>ld-linux.so.2</filename>) loads DSOs into memory by using |
| mmap, this means that DSOs are no longer loaded at predictable |
| addresses, so making it harder to attack software by using buffer |
| overflows. It also attempts to relocate certain binaries into a |
| special low area of memory known as the ASCII armor so making it |
| harder to jump into them when using string based attacks. |
| </para> |
| |
| <para> |
| Prelink is a technology that enhances startup times by precalculating |
| ELF global offset tables then saving the results inside the native |
| binaries themselves. By grid fitting each DSO into the address space, |
| the dynamic linker does not have to perform as many relocations so |
| allowing applications that heavily rely on dynamic linkage to be |
| loaded into memory much quicker. Complex C++ applications such as |
| Mozilla, OpenOffice and KDE can especially benefit from this |
| technique. |
| </para> |
| |
| <para> |
| The 4G VM split patch was developed by Ingo Molnar. It gives the Linux |
| kernel its own address space, thereby allowing processes to access the |
| maximum addressable amount of memory on a 32-bit machine: 4 |
| gigabytes. It allows people with lots of RAM to fully utilise that in |
| any given process at the cost of performance: the reason behind giving |
| the kernel a part of each processes address space was to avoid the |
| overhead of switching on each syscall. |
| </para> |
| |
| <para> |
| Each of these changes alter the address space in a way incompatible |
| with Windows. Prelink and exec-shield mean that the libraries Wine |
| uses can be placed at any point in the address space: typically this |
| meant that a library was sitting in the region that the EXE you wanted |
| to run had to be loaded (remember that unlike DLLs, EXE files cannot |
| be moved around in memory). The 4G VM split means that programs could |
| receive pointers to the top gigabyte of address space which some are |
| not prepared for (they may store extra information in the high bits of |
| a pointer, for instance). In particular, in combination with |
| exec-shield this one is especially deadly as it's possible the process |
| heap could be allocated beyond |
| <constant>ADDRESS_SPACE_LIMIT</constant> which causes Wine |
| initialization to fail. |
| </para> |
| |
| <para> |
| The solution to these problems is for Wine to reserve particular parts |
| of the address space so that areas that we don't want the system to |
| use will be avoided. We later on (re/de)allocate those areas as |
| needed. One problem is that some of these mappings are put in place |
| automatically by the dynamic linker: for instance any libraries that |
| Wine is linked to (like <filename>libc</filename>, |
| <filename>libwine</filename>, <filename>libpthread</filename> etc) |
| will be mapped into memory before Wine even gets control. In order to |
| solve that, Wine overrides the default ELF initialization sequence at |
| a low level and reserves the needed areas by using direct syscalls |
| into the kernel (ie without linking against any other code to do it) |
| before restarting the standard initialization and letting the dynamic |
| linker continue. This is referred to as the preloader and is found in |
| <filename>loader/preloader.c</filename>. |
| </para> |
| |
| <para> |
| Once the usual ELF boot sequence has been completed, some native |
| libraries may well have been mapped above the 3gig limit: however, |
| this doesn't matter as 3G is a Windows limit, not a Linux limit. We |
| still have to prevent the system from allocating anything else above |
| there (like the heap or other DLLs) though so Wine performs a binary |
| search over the upper gig of address space in order to iteratively |
| fill in the holes with <constant>MAP_NORESERVE</constant> mappings so |
| the address space is allocated but the memory to actually back it is |
| not. This code can be found in |
| <filename>libs/wine/mmap.c</filename>:<function>reserve_area</function>. |
| </para> |
| </sect2> |
| </sect1> |
| |
| <sect1> |
| <title>Multi-processing in Wine</title> |
| <para> |
| Let's take a closer look at the way Wine loads and run processes in memory. |
| </para> |
| <sect2> |
| <title>Starting a process from command line</title> |
| <para> |
| When starting a Wine process from command line (we'll get later on to |
| the differences between NE, PE and Winelib executables), there are a |
| couple of things Wine need to do first. A first executable is run to |
| check the threading model of the underlying OS (see |
| <xref linkend="threading"> for the details) and will start the real |
| Wine loader corresponding to the choosen threading model. |
| </para> |
| <para> |
| Then Wine graps a few elements from the Unix world: the environment, |
| the program arguments. Then the <filename>ntdll.dll.so</filename> is |
| loaded into memory using the standard shared library dynamic |
| loader. When loaded, <filename>NTDLL</filename> will mainly first |
| create a decent Windows environment: |
| <itemizedlist> |
| <listitem> |
| <para> |
| create a PEB (Process Environment Block) and a TEB (Thread |
| Environment Block). |
| </para> |
| </listitem> |
| <listitem> |
| <para> |
| set up the connection to the Wine server - and eventually |
| launching the Wine server if none runs |
| </para> |
| </listitem> |
| <listitem> |
| <para>create the process heap</para> |
| </listitem> |
| </itemizedlist> |
| </para> |
| <para> |
| Then <filename>Kernel32</filename> is loaded (but now using the |
| Windows dynamic loading capabilities) and a Wine specific entry point |
| is called <function>__wine_kernel_init</function>. This function will |
| actually handle all the logic of the process loading and execution, |
| and will never return from it's call. |
| </para> |
| <para> |
| <function>__wine_kernel_init</function> will undergo the following |
| tasks: |
| <itemizedlist> |
| <listitem> |
| <para> |
| initialization of program arguments from Unix program arguments |
| </para> |
| </listitem> |
| <listitem> |
| <para>lookup of executable in the file system</para> |
| </listitem> |
| <listitem> |
| <para> |
| If the file is not found, then an error is printed and the Wine |
| loader stops. |
| </para> |
| </listitem> |
| <listitem> |
| <para> |
| We'll cover the non-PE file type later on, so assume for now |
| it's a PE file. The PE module is loaded in memory using the |
| same mechanisms as for a native DLLs (mainly mapping the file |
| data and code sections into memory, and handling relocation if |
| needed). Note that the dependencies on the module are not |
| resolved at this point. |
| </para> |
| </listitem> |
| <listitem> |
| <para> |
| A new stack is created, which size is given in the PE header, |
| and this stack is made the one of the running thread (which is |
| still the only one in the process). The stack used at startup |
| will no longer be used. |
| </para> |
| </listitem> |
| <listitem> |
| <para> |
| Which this new stack, |
| <function>ntdll.LdrInitializeThunk</function> is called which |
| performs the remaining initialization parts, including resolving |
| all the DLL imports on the PE module, and doing the init of the |
| TLS slots. |
| </para> |
| </listitem> |
| <listitem> |
| <para> |
| Control can now be passed to the <function>EntryPoint</function> |
| of the PE module, which will let the executable run. |
| </para> |
| </listitem> |
| </itemizedlist> |
| </para> |
| </sect2> |
| <sect2> |
| <title>Creating a child process from a running process</title> |
| <para> |
| The steps used are closely link to what is done in the previous case. |
| </para> |
| <para> |
| There are however a few points to look at a bit more closely. The |
| inner implementation creates the child process using the |
| <function>fork()</function> and <function>exec()</function> |
| calls. This means that we don't need to check again for the threading |
| model, we can use what the parent (or the grand-parent process...) |
| started from command line has found. |
| </para> |
| <para> |
| The Win32 process creation allows to pass a lot of information between |
| the parent and the child. This includes object handles, windows title, |
| console parameters, environment strings... Wine makes use of both the |
| standard Unix inheritance mechanisms (for environment for example) and |
| the Wine server (to pass from parent to child a chunk of data |
| containing the relevant information). |
| </para> |
| <para> |
| The previously described loading mechanism will check in the Wine |
| server if such a chunk exists, and, if so, will perform the relevant |
| initialization. |
| </para> |
| <para> |
| Some further synchronization is also put in place: a parent will wait |
| until the child has started, or has failed. The Wine server is also |
| used to perform those tasks. |
| </para> |
| </sect2> |
| <sect2> |
| <title>Starting a Winelib process</title> |
| <para> |
| Before going into the gory details, let's first go back to what a |
| Winelib application is. It can be either a regular Unix executable, or |
| a more specific Wine beast. This later form in fact creates two files |
| for a given executable (say <filename>foo.exe</filename>). The first |
| one, named <filename>foo</filename> will be a symbolic link to the |
| Wine loader (<filename>wine</filename>). The second one, named |
| <filename>foo.exe.so</filename>, is the equivalent of the |
| <filename>.dll.so</filename> files we've already described for |
| DLLs. As in Windows, an executable is, among other things, a module |
| with its import and export information, as any DLL, it makes sense |
| Wine uses the same mechanisms for loading native executables and |
| DLLs. |
| </para> |
| <para> |
| When starting a Winelib application from the command line (say with |
| <command>foo arg1 arg2</command>), the Unix shell will execute |
| <command>foo</command> as a Unix executable. Since this is in fact the |
| Wine loader, Wine will fire up. However, will notice that it hasn't |
| been started as <command>wine</command> but as <command>foo</command>, |
| and hence, will try to load (using Unix shared library mechanism) the |
| second file <filename>foo.exe.so</filename>. Wine will recognize a 32 |
| bit module (with its descriptor) embedded in the shared library, and |
| once the shared library loaded, it will proceed the same path as when |
| loading a standard native PE executable. |
| </para> |
| <para> |
| Wine needs to implement this second form of executable in order to |
| maintain the order of initialization of some elements in the |
| executable. One particular issue is when dealing with global C++ |
| objects. In standard Unix executable, the call of the constructor to |
| such objects is stored in the specific section of the executable |
| (<function>.init</function> not to name it). All constructors in this |
| section are called before the <function>main()</function> or |
| <function>WinMain</function> function is called. Creating a Wine |
| executable using the first form mentionned above will let those |
| constructors being called before Wine gets a chance to initialize |
| itself. So, any constructor using a Windows API will fail, because |
| Wine infrastructure isn't in place. The use of the second form for |
| Winelib executables ensures that we do the initialization using the |
| following steps: |
| <itemizedlist> |
| <listitem> |
| <para> |
| initialize the Wine infrastructure |
| </para> |
| </listitem> |
| <listitem> |
| <para> |
| load the executable into memory |
| </para> |
| </listitem> |
| <listitem> |
| <para> |
| handle the import sections for the executable |
| </para> |
| </listitem> |
| <listitem> |
| <para> |
| call the global object constructors (if any). They now can |
| properly call the Windows APIs |
| </para> |
| </listitem> |
| <listitem> |
| <para> |
| call the executable entry point |
| </para> |
| </listitem> |
| </itemizedlist> |
| </para> |
| <para> |
| The attentive reader would have noted that the resolution of imports |
| for the executable is done, as for a DLL, when the executable/DLL |
| descriptor is registered. However, this is done also by adding a |
| specific constructor in the <function>.init</function> section. For |
| the above describe scheme to function properly, this constructor must |
| be the first constructor to be called, before all the other |
| constructors, generated by the executable itself. The Wine build chain |
| takes care of that, and also generating the executable/DLL descriptor |
| for the Winelib executable. |
| </para> |
| </sect2> |
| </sect1> |
| |
| <sect1 id="threading"> |
| <title>Multi-threading in Wine</title> |
| |
| <para> |
| This section will assume you understand the basics of multithreading. If |
| not there are plenty of good tutorials available on the net to get you |
| started. |
| </para> |
| |
| <para> |
| Threading in Wine is somewhat complex due to several factors. The first |
| is the advanced level of multithreading support provided by Windows - |
| there are far more threading related constructs available in Win32 than |
| the Linux equivalent (pthreads). The second is the need to be able to |
| map Win32 threads to native Linux threads which provides us with |
| benefits like having the kernel schedule them without our |
| intervention. While it's possible to implement threading entirely |
| without kernel support, doing so is not desirable on most platforms that |
| Wine runs on. |
| </para> |
| |
| <sect2> |
| <title>Threading support in Win32</title> |
| |
| <para> |
| Win32 is an unusually thread friendly API. Not only is it entirely |
| thread safe, but it provides many different facilities for working |
| with threads. These range from the basics such as starting and |
| stopping threads, to the extremely complex such as injecting threads |
| into other processes and COM inter-thread marshalling. |
| </para> |
| |
| <para> |
| One of the primary challenges of writing Wine code therefore is |
| ensuring that all our DLLs are thread safe, free of race conditions |
| and so on. This isn't simple - don't be afraid to ask if you aren't |
| sure whether a piece of code is thread safe or not! |
| </para> |
| |
| <para> |
| Win32 provides many different ways you can make your code thread safe |
| however the most common are <emphasis>critical section</emphasis> and |
| the <emphasis>interlocked functions</emphasis>. Critical sections are |
| a type of mutex designed to protect a geographic area of code. If you |
| don't want multiple threads running in a piece of code at once, you |
| can protect them with calls to |
| <function>EnterCriticalSection()</function> and |
| <function>LeaveCriticalSection()</function>. The first call to |
| <function>EnterCriticalSection()</function> by a thread will lock the |
| section and continue without stopping. If another thread calls it then |
| it will block until the original thread calls |
| <function>LeaveCriticalSection()</function> again. |
| </para> |
| |
| <para> |
| It is therefore vitally important that if you use critical sections to |
| make some code thread-safe, that you check every possible codepath out |
| of the code to ensure that any held sections are left. Code like this: |
| </para> |
| |
| <programlisting> |
| if (res != ERROR_SUCCESS) return res; |
| </programlisting> |
| |
| <para> |
| is extremely suspect in a function that also contains a call to |
| <function>EnterCriticalSection()</function>. Be careful. |
| </para> |
| |
| <para> |
| If a thread blocks while waiting for another thread to leave a |
| critical section, you will see an error from the |
| <function>RtlpWaitForCriticalSection()</function> function, along with |
| a note of which thread is holding the lock. This only appears after a |
| certain timeout, normally a few seconds. It's possible the thread |
| holding the lock is just being really slow which is why Wine won't |
| terminate the app like a non-checked build of Windows would, but the |
| most common cause is that for some reason a thread forgot to call |
| <function>LeaveCriticalSection()</function>, or died while holding the |
| lock (perhaps because it was in turn waiting for another lock). This |
| doesn't just happen in Wine code: a deadlock while waiting for a |
| critical section could be due to a bug in the app triggered by a |
| slight difference in the emulation. |
| </para> |
| |
| <para> |
| Another popular mechanism available is the use of functions like |
| <function>InterlockedIncrement()</function> |
| and <function>InterlockedExchange()</function>. These make use of native |
| CPU abilities to execute a single instruction while ensuring any other |
| processors on the system cannot access memory, and allow you to do |
| common operations like add/remove/check a variable in thread-safe code |
| without holding a mutex. These are useful for reference counting |
| especially in free-threaded (thread safe) COM objects. |
| </para> |
| |
| <para> |
| Finally, the usage of TLS slots are also popular. TLS stands for |
| thread-local storage, and is a set of slots scoped local to a thread |
| which you can store pointers in. Look on MSDN for the |
| <function>TlsAlloc()</function> function to learn more about the Win32 |
| implementation of this. Essentially, the contents of a given slot will |
| be different in each thread, so you can use this to store data that is |
| only meaningful in the context of a single thread. On recent versions |
| of Linux the __thread keyword provides a convenient interface to this |
| functionality - a more portable API is exposed in the pthread |
| library. However, these facilities are not used by Wine, rather, we |
| implement Win32 TLS entirely ourselves. |
| </para> |
| </sect2> |
| |
| <sect2> |
| <title>POSIX threading vs. kernel threading</title> |
| |
| <para> |
| Wine runs in one of two modes: either pthreads (posix threading) or |
| kthreads (kernel threading). This section explains the differences |
| between them. The one that is used is automatically selected on |
| startup by a small test program which then execs the correct binary, |
| either <command>wine-kthread</command> or |
| <command>wine-pthread</command>. On NPTL-enabled systems pthreads |
| will be used, and on older non-NPTL systems kthreads is selected. |
| </para> |
| |
| <para> |
| Let's start with a bit of history. Back in the dark ages when Wine's |
| threading support was first implemented a problem was faced - Windows |
| had much more capable threading APIs than Linux did. This presented a |
| problem - Wine works either by reimplementing an API entirely or by |
| mapping it onto the underlying systems equivalent. How could Win32 |
| threading be implemented using a library which did not have all the |
| needed features? The answer, of course, was that it couldn't be. |
| </para> |
| |
| <para> |
| On Linux the pthreads interface is used to start, stop and control |
| threads. The pthreads library in turn is based on top of so-called |
| "kernel threads" which are created using the |
| <function>clone(2)</function> syscall. Pthreads provides a nicer (more |
| portable) interface to this functionality and also provides APIs for |
| controlling mutexes. There is a <ulink |
| url="http://www.llnl.gov/computing/tutorials/pthreads/"> good |
| tutorial on pthreads</ulink> available if you want to learn more. |
| </para> |
| |
| <para> |
| As pthreads did not provide the necessary semantics to implement Win32 |
| threading, the decision was made to implement Win32 threading on top |
| of the underlying kernel threads by using syscalls like |
| <function>clone()</function> directly. This provided maximum |
| flexibility and allowed a correct implementation but caused some bad |
| side effects. Most notably, all the userland Linux APIs assumed that |
| the user was utilising the pthreads library. Some only enabled thread |
| safety when they detected that pthreads was in use - this is true of |
| glibc, for instance. Worse, pthreads and pure kernel threads had |
| strange interactions when run in the same process yet some libraries |
| used by Wine used pthreads internally. Throw in source code porting |
| using Winelib - where you have both UNIX and Win32 code in the same |
| process - and chaos was the result. |
| </para> |
| |
| <para> |
| The solution was simple yet ingenious: Wine would provide its own |
| implementation of the pthread library <emphasis>inside</emphasis> its |
| own binary. Due to the semantics of ELF symbol scoping, this would |
| cause Wine's own implementation to override any implementation loaded |
| later on (like the real libpthread.so). Therefore, any calls to the |
| pthread APIs in external libraries would be linked to Wine's instead |
| of the system's pthreads library, and Wine implemented pthreads by |
| using the standard Windows threading APIs it in turn implemented |
| itself. |
| </para> |
| |
| <para> |
| As a result, libraries that only became thread-safe in the presence of |
| a loaded pthreads implementation would now do so, and any external |
| code that used pthreads would actually end up creating Win32 threads |
| that Wine was aware of and controlled. This worked quite nicely for a |
| long time, even though it required doing some extremely un-kosher |
| things like overriding internal libc structures and functions. That |
| is, it worked until NPTL was developed at which point the underlying |
| thread implementation on Linux changed dramatically. |
| </para> |
| |
| <para> |
| The fake pthread implementation can be found in |
| <filename>loader/kthread.c</filename>, which is used to |
| produce the <command>wine-kthread</command> binary. In contrast, |
| <filename>loader/pthread.c</filename> produces the |
| <command>wine-pthread</command> binary which is used on newer NPTL |
| systems. |
| </para> |
| |
| <para> |
| NPTL is a new threading subsystem for Linux that hugely improves its |
| performance and flexibility. By allowing threads to become much more |
| scalable and adding new pthread APIs, NPTL made Linux competitive with |
| Windows in the multi-threaded world. Unfortunately it also broke many |
| assumptions made by Wine (as well as other applications such as the |
| Sun JVM and RealPlayer) in the process. |
| </para> |
| |
| <para> |
| There was, however, some good news. NPTL made Linux threading powerful |
| enough that Win32 threads could now be implemented on top of pthreads |
| like any other normal application. There would no longer be problems |
| with mixing win32-kthreads and pthreads created by external libraries, |
| and no need to override glibc internals. As you can see from the |
| relative sizes of the <filename>loader/kthread.c</filename> and |
| <filename>loader/pthread.c</filename> files, the difference in code |
| complexity is considerable. NPTL also made several other semantic |
| changes to things such as signal delivery so changes were required in |
| many different places in Wine. |
| </para> |
| |
| <para> |
| On non-Linux systems the threading interface is typically not powerful |
| enough to replicate the semantics Win32 applications expect and so |
| kthreads with the pthread overrides are used. |
| </para> |
| </sect2> |
| |
| <sect2> |
| <title>The Win32 thread environment</title> |
| |
| <para> |
| All Win32 code, whether from a native EXE/DLL or in Wine itself, |
| expects certain constructs to be present in its environment. This |
| section explores what those constructs are and how Wine sets them |
| up. The lack of this environment is one thing that makes it hard to |
| use Wine code directly from standard Linux applications - in order to |
| interact with Win32 code a thread must first be |
| "adopted" by Wine. |
| </para> |
| |
| <para> |
| The first thing Win32 code requires is the |
| <emphasis>TEB</emphasis> or "Thread Environment Block". This is an |
| internal (undocumented) Windows structure associated with every thread |
| which stores a variety of things such as TLS slots, a pointer to the |
| threads message queue, the last error code and so on. You can see the |
| definition of the TEB in <filename>include/thread.h</filename>, or at |
| least what we know of it so far. Being internal and subject to change, |
| the layout of the TEB has had to be reverse engineered from scratch. |
| </para> |
| |
| <para> |
| A pointer to the TEB is stored in the %fs register and can be accessed |
| using <function>NtCurrentTeb()</function> from within Wine code. %fs |
| actually stores a selector, and setting it therefore requires |
| modifying the processes local descriptor table (LDT) - the code to do |
| this is in <filename>lib/wine/ldt.c</filename>. |
| </para> |
| |
| <para> |
| The TEB is required by nearly all Win32 code run in the Wine |
| environment, as any <command>wineserver</command> RPC will use it, |
| which in turn implies that any code which could possibly block for |
| instance by using a critical section) needs it. The TEB also holds the |
| SEH exception handler chain as the first element, so if disassembling |
| you see code like this: |
| </para> |
| |
| <programlisting>movl %esp, %fs:0</programlisting> |
| |
| <para> |
| ... then you are seeing the program set up an SEH handler frame. All |
| threads must have at least one SEH entry, which normally points to the |
| backstop handler which is ultimately responsible for popping up the |
| all-too-familiar This program has performed an illegal operation and |
| will be terminated" message. On Wine we just drop straight into the |
| debugger. A full description of SEH is out of the scope of this |
| section, however there are some good articles in MSJ if you are |
| interested. |
| </para> |
| |
| <para> |
| All Win32-aware threads must have a <command>wineserver</command> |
| connection. Many different APIs require the ability to communicate |
| with the <command>wineserver</command>. In turn, the |
| <command>wineserver</command> must be aware of Win32 threads in order |
| to be able to accurately report information to other parts of the |
| program and do things like route inter-thread messages, dispatch APCs |
| (asynchronous procedure calls) and so on. Therefore a part of thread |
| initialization is initializing the thread server-side. The result is |
| not only correct information in the server, but a set of file |
| descriptors the thread can use to communicate with the server - the |
| request fd, reply fd and wait fd (used for blocking). |
| </para> |
| </sect2> |
| </sect1> |
| |
| <sect1 id="seh"> |
| <title>Structured Exception Handling</title> |
| |
| <para> |
| Structured Exception Handling (or SEH) is an implementation of |
| exceptions inside the Windows core. It allows code written in different |
| languages to throw exceptions across DLL boundaries, and Windows reports |
| various errors like access violations by throwing them. This section |
| looks at how it works, and how it's implemented in Wine. |
| </para> |
| |
| <sect2> |
| <title>How SEH works</title> |
| |
| <para> |
| SEH is based on embedding |
| <structname>EXCEPTION_REGISTRATION_RECORD</structname> structures in |
| the stack. Together they form a linked list rooted at offset zero in |
| the TEB (see the threading section if you don't know what this is). A |
| registration record points to a handler function, and when an |
| exception is thrown the handlers are executed in turn. Each handler |
| returns a code, and they can elect to either continue through the |
| handler chain or it can handle the exception and then restart the |
| program. This is referred to as unwinding the stack. After each |
| handler is called it's popped off the chain. |
| </para> |
| |
| <para> |
| Before the system begins unwinding the stack, it runs vectored |
| handlers. This is an extension to SEH available in Windows XP, and |
| allows registered functions to get a first chance to watch or deal |
| with any exceptions thrown in the entire program, from any thread. |
| </para> |
| |
| <para> |
| A thrown exception is represented by an |
| <structname>EXCEPTION_RECORD</structname> structure. It consists of a |
| code, flags, an address and an arbitrary number of <type>DWORD</type> |
| parameters. Language runtimes can use these parameters to associate |
| language-specific information with the exception. |
| </para> |
| |
| <para> |
| Exceptions can be triggered by many things. They can be thrown |
| explicitly by using the RaiseException API, or they can be triggered |
| by a crash (ie, translated from a signal). They may be used internally |
| by a language runtime to implement language-specific exceptions. They |
| can also be thrown across DCOM connections. |
| </para> |
| |
| <para> |
| Visual C++ has various extensions to SEH which it uses to implement, |
| eg, object destruction on stack unwind as well as the ability to throw |
| arbitrary types. The code for this is in |
| <filename>dlls/msvcrt/except.c</filename> |
| </para> |
| |
| </sect2> |
| |
| <sect2> |
| <title>Translating signals to exceptions</title> |
| |
| <para> |
| In Windows, compilers are expected to use the system exception |
| interface, and the kernel itself uses the same interface to |
| dynamically insert exceptions into a running program. By contrast on |
| Linux the exception ABI is implemented at the compiler level |
| (inside GCC and the linker) and the kernel tells a thread of |
| exceptional events by sending <emphasis>signals</emphasis>. The |
| language runtime may or may not translate these signals into native |
| exceptions, but whatever happens the kernel does not care. |
| </para> |
| |
| <para> |
| You may think that if an app crashes, it's game over and it really |
| shouldn't matter how Wine handles this. It's what you might |
| intuitively guess, but you'd be wrong. In fact some Windows programs |
| expect to be able to crash themselves and recover later without the |
| user noticing, some contain buggy binary-only components from third |
| parties and use SEH to swallow crashes, and still others execute |
| priviledged (kernel-level) instructions and expect it to work. In |
| fact, at least one set of APIs (the <function>IsBad*Ptr()</function> |
| series) can only be implemented by performing an operation that may |
| crash and returning <constant>TRUE</constant> if it does, and |
| <constant>FALSE</constant> if it doesn't! So, Wine needs to not only |
| implement the SEH infrastructure but also translate Unix signals into |
| SEH exceptions. |
| </para> |
| |
| <para> |
| The code to translate signals into exceptions is a part of |
| <filename>NTDLL</filename>, and can be found in |
| <filename>dlls/ntdll/signal_i386.c</filename>. This file sets up |
| handlers for various signals during Wine startup, and for the ones |
| that indicate exceptional conditions translates them into |
| exceptions. Some signals are used by Wine internally and have nothing |
| to do with SEH. |
| </para> |
| |
| <para> |
| Signal handlers in Wine run on their own stack. Each thread has its |
| own signal stack which resides 4k after the TEB. This is important for |
| a couple of reasons. Firstly, because there's no guarantee that the |
| app thread which triggered the signal has enough stack space for the |
| Wine signal handling code. In Windows, if a thread hits the limits of |
| its stack it triggers a fault on the stack guard page. The language |
| runtime can use this to grow the stack if it wants to. |
| |
| <!-- fixme: is it really the language runtime that does this? i |
| can't find any code in Wine to reallocate the stack on |
| STATUS_GUARD_PAGE_VIOLATION --> |
| |
| However, because a guard page violation is just a regular segfault to |
| the kernel, that would lead to a nested signal handler and that gets |
| messy really quick so we disallow that in Wine. Secondly, setting up |
| the exception to throw requires modifying the stack of the thread |
| which triggered it, which is quite hard to do when you're still |
| running on it. |
| </para> |
| |
| <para> |
| Windows exceptions typically contain more information than the Unix |
| standard APIs provide. For instance, a |
| <constant>STATUS_ACCESS_VIOLATION</constant> exception |
| (<constant>0xC0000005</constant>) structure contains the faulting |
| address, whereas a standard Unix <constant>SIGSEGV</constant> just |
| tells the app that it crashed. Usually this information is passed as |
| an extra parameter to the signal handler, however its location and |
| contents vary between kernels (BSD, Solaris, etc). This data is |
| provided in a <structname>SIGCONTEXT</structname> structure, and on |
| entry to the signal handler it contains the register state of the CPU |
| before the signal was sent. Modifying it will cause the kernel to |
| adjust the context before restarting the thread. |
| </para> |
| </sect2> |
| </sect1> |
| |
| <sect1> |
| <title>File management</title> |
| <para> |
| With time, Windows API comes closer to the old Unix paradigm "Everything |
| is a file". Therefore, this whole section dedicated to file management |
| will cover firstly the file management, but also some other objects like |
| directories, and even devices, which are manipulated in Windows in a |
| rather coherent way. We'll see later on some other objects fitting |
| (more or less) in this picture (pipes or consoles to name a few). |
| </para> |
| |
| <para> |
| First of all, Wine, while implementing the file interface from Windows, |
| needs to maps a file name (expressed in the Windows world) onto a file |
| name in the Unix world. This encompasses several aspects: how to map |
| the file names, how to map access rights (both on files and |
| directories), how to map physical devices (hardisks, but also other |
| devices - like serial or parallel interfaces - and even VxDs). |
| </para> |
| |
| <sect2> |
| <title>Various Windows formats for file names</title> |
| <para> |
| Let's first review a bit the various forms Windows uses when it comes |
| to file names. |
| </para> |
| |
| <sect3> |
| <title>The DOS inheritance</title> |
| |
| <para> |
| At the beginning was DOS, where each file has to sit on a drive, |
| called from a single letter. For separating device names from |
| directory or file names, a ':' was appended to this single letter, |
| hence giving the (in)-famous <filename>C:</filename> drive |
| designations. Another great invention was to use some fixed names |
| for accessing devices: not only where these named fixed, in a way |
| you couldn't change the name if you'd wish to, but also, they were |
| insensible to the location where you were using them. For example, |
| it's well known that <filename>COM1</filename> designates the first |
| serial port, but it's also true that |
| <filename>c:\foo\bar\com1</filename> also designates the first |
| serial port. It's still true today: on XP, you still cannot name a |
| file <filename>COM1</filename>, whatever the directory!!! |
| </para> |
| <para> |
| Well later on (with Windows 95), Microsoft decided to overcome some |
| little details in file names: this included being able to get out of |
| the 8+3 format (8 letters for the name, 3 letters for the |
| extension), and so being able to use "long names" (that's the |
| "official" naming; as you can guess, the 8+3 format is a short |
| name), and also to use very strange characters in a file name (like |
| a space, or even a '.'). You could then name a file |
| <filename>My File V0.1.txt</filename>, instead of |
| <filename>myfile01.txt</filename>. Just to keep on the fun side of |
| things, for many years the format used on the disk itself for |
| storing the names has been the short name as the real one and to use |
| some tricky aliasing techniques to store the long name. When some |
| newer disk file systems have been introduced (NTFS with NT), in |
| replacement of the old FAT system (which had little evolved since |
| the first days of DOS), the long name became the real name while the |
| short name took the alias role. |
| </para> |
| <para> |
| Windows also started to support mounting network shares, and see |
| them as they were a local disk (through a specific drive letter). |
| The way it has been done changed along the years, so we won't go |
| into all the details (especially on the DOS and Win9x side). |
| </para> |
| </sect3> |
| |
| <sect3> |
| <title>The NT way</title> |
| <para> |
| The introduction of NT allowed a deep change in the ways DOS had |
| been handling devices: |
| <itemizedlist> |
| <listitem> |
| <para> |
| There's no longer a forest of DOS drive letters (even if the |
| <command>assign</command> was a way to create symbolic links |
| in the forest), but a single hierarchical space. |
| </para> |
| </listitem> |
| <listitem> |
| <para> |
| This hierarchy includes several distinct elements. For |
| example, <filename>\Device\Hardisk0\Partition0</filename> |
| refers to the first partition on the first physical hard disk |
| of the system. |
| </para> |
| </listitem> |
| <listitem> |
| <para> |
| This hierarchy covers way more than just the files and drives |
| related objects, but most of the objects in the system. We'll |
| only cover here the file related part. |
| </para> |
| </listitem> |
| <listitem> |
| <para> |
| This hierarchy is not directly accessible for the Win32 API, |
| but only the <filename>NTDLL</filename> API. The Win32 API |
| only allows to manipulate part of this hierarchy (the rest |
| being hidden from the Win32 API). Of course, the part you see |
| from Win32 API looks very similar to the one that DOS |
| provided. |
| </para> |
| </listitem> |
| <listitem> |
| <para> |
| Mounting a disk is performed by creating a symbol link in this |
| hierarchy from <filename>\Global??\C:</filename> (the name |
| seen from the Win32 API) to |
| <filename>\Device\Harddiskvolume1</filename> which determines |
| the partition on a physical disk where C: is going to be seen. |
| </para> |
| </listitem> |
| <listitem> |
| <para> |
| Network shares are also accessible through a symbol link. |
| However in this case, a symbol link is created from |
| <filename>\Global??\UNC\host\share\</filename> for the share |
| <filename>share</filename> on the machine |
| <filename>host</filename>) to what's called a network |
| redirector, and which will take care of 1/ the connection to |
| the remote share, 2/ handling with that remote share the rest |
| of the path (after the name of the server, and the name of the |
| share on that server). |
| </para> |
| <note> |
| <para> |
| In NT naming convention, <filename>\Global??</filename> can |
| also be called <filename>\??</filename> to shorten the |
| access. |
| </para> |
| </note> |
| </listitem> |
| </itemizedlist> |
| </para> |
| <para> |
| All of these things, make the NT system pretty much more flexible |
| (you can add new types of filesystems if you want), you provide a |
| unique name space for all objects, and most operations boil down to |
| creating relationship between different objects. |
| </para> |
| </sect3> |
| |
| <sect3> |
| <title>Wrap up</title> |
| <para> |
| Let's end this chapter about files in Windows with a review of the |
| different formats used for file names: |
| <itemizedlist> |
| <listitem> |
| <para><filename>c:\foo\bar</filename> is a full path.</para> |
| </listitem> |
| <listitem> |
| <para> |
| <filename>\foo\bar</filename> is an absolute path; the full |
| path is created by appending the default drive (ie. the drive |
| of the current directory). |
| </para> |
| </listitem> |
| <listitem> |
| <para> |
| <filename>bar</filename> is a relative path; the full path is |
| created by adding the current directory. |
| </para> |
| </listitem> |
| <listitem> |
| <para> |
| <filename>c:bar</filename> is a drive relative path. Note |
| that the case where <filename>c:</filename> is the drive of |
| the current directory is rather easy; it's implemented the |
| same way as the case just below (relative path). In the rest |
| of this chapter, drive relative path will only cover the case |
| where the drive in the path isn't the drive of the default |
| directory. The resolution of this to a full pathname defers |
| according to the version of Windows, and some parameters. |
| Let's take some time browsing through these issues. On |
| Windows 9x (as well as on DOS), the system maintains a process |
| wide set of default directories per drive. Hence, in this |
| case, it will resolve <filename>c:bar</filename> to the |
| default directory on drive <filename>c:</filename> plus file |
| <filename>bar</filename>. Of course, the default per drive |
| directory is updated each time a new current directory is set |
| (only the current directory of the drive specified is |
| modified). On Windows NT, things differ a bit. Since NT |
| implements a namespace for file closer to a single tree |
| (instead of 26 drives), having a current directory per drive |
| is a bit ackward. Hence, Windows NT default behavior is to |
| have only one current directory across all drives (in fact, a |
| current directory expressed in the global tree) - this |
| directory is of course related to a given process -, |
| <filename>c:bar</filename> is resolved this way: |
| <itemizedlist> |
| <listitem> |
| <para> |
| If <filename>c:</filename> is the drive of the default |
| directory, the final path is the current directory plus |
| <filename>bar</filename>. |
| </para> |
| </listitem> |
| <listitem> |
| <para> |
| Otherwise it's resolved into |
| <filename>c:\bar</filename>. |
| </para> |
| </listitem> |
| <listitem> |
| <para> |
| In order to bridge the gap between the two |
| implementations (Windows 9x and NT), NT adds a bit of |
| complexity on the second case. If the |
| <envar>=C:</envar> environment variable is defined, then |
| it's value is used as a default directory for drive |
| <filename>C:</filename>. This is handy, for example, |
| when writing a DOS shell, where having a current drive |
| per drive is still implemented, even on NT. This |
| mechanism (through environment variables) is implemented |
| on <command>CMD.EXE</command>, where those variables are |
| set when you change directories with the |
| <command>cd</command>. Since environment variables are |
| inherited at process creation, the current directories |
| settings are inherited by child processes, hence |
| mimicing the behavior of the old DOS shell. There's no |
| mechanism (in <filename>NTDLL</filename> or |
| <filename>KERNEL32</filename>) to set up, when current |
| directory changes, the relevant environment variables. |
| This behavior is clearly band-aid, not a full featured |
| extension of current directory behavior. |
| </para> |
| </listitem> |
| </itemizedlist> |
| Wine fully implements all those behaviors (the Windows 9x vs |
| NT ones are triggered by the version flag in Wine). |
| </para> |
| </listitem> |
| <listitem> |
| <para> |
| <filename>\\host\share</filename> is <firstterm>UNC</firstterm> |
| (Universal Naming Convention) path, ie. represents a file on a |
| remote share. |
| </para> |
| </listitem> |
| <listitem> |
| <para> |
| <filename>\\.\device</filename> denotes a physical device |
| installed in the system (as seen from the Win32 subsystem). A |
| standard NT system will map it to the |
| <filename>\??\device</filename> NT path. Then, as a standard |
| configuration, <filename>\??\device</filename> is likely to be |
| a link to in a physical device described and hooked into the |
| <filename>\Device\</filename> tree. For example, |
| <filename>COM1</filename> is a link to |
| <filename>\Device\Serial0</filename>. |
| </para> |
| </listitem> |
| <listitem> |
| <para> |
| On some versions of Windows, paths were limited to |
| <constant>MAX_PATH</constant> characters. To circumvent this, |
| Microsoft allowed paths to be <constant>32,767</constant> |
| characters long, under the conditions that the path is |
| expressed in Unicode (no Ansi version), and that the path is |
| prefixed with <filename>\\?\</filename>. This convention is |
| applicable to any of the cases described above. |
| </para> |
| </listitem> |
| </itemizedlist> |
| </para> |
| <para> |
| To summarize, what we've discussed so, let's put everything into a |
| single table... |
| <table> |
| <title>DOS, Win32 and NT paths equivalences</title> |
| <tgroup cols="3" align="left"> |
| <thead> |
| <row> |
| <entry>Type of path</entry> |
| <entry>Win32 example</entry> |
| <entry>NT equivalent</entry> |
| <entry>Rule to construct</entry> |
| </row> |
| </thead> |
| <tbody> |
| <row> |
| <entry>Full path</entry> |
| <entry><filename>c:\foo\bar.txt</filename></entry> |
| <entry><filename>\Global??\C:\foo\bar.txt</filename></entry> |
| <entry>Simple concatenation</entry> |
| </row> |
| <row> |
| <entry>Absolute path</entry> |
| <entry><filename>\foo\bar.txt</filename></entry> |
| <entry><filename>\Global??\J:\foo\bar.txt</filename></entry> |
| <entry> |
| Simple concatenation using the drive of the default |
| directory (here J:) |
| </entry> |
| </row> |
| <row> |
| <entry>Relative path</entry> |
| <entry><filename>gee\bar.txt</filename></entry> |
| <entry> |
| <filename> |
| \Global??\J:\mydir\mysubdir\gee\bar.txt |
| </filename> |
| </entry> |
| <entry> |
| Simple concatenation using the default directory |
| (here <filename>J:\mydir\mysubdir</filename>) |
| </entry> |
| </row> |
| <row> |
| <entry>Drive relative path</entry> |
| <entry><filename>j:gee\bar.txt</filename></entry> |
| <entry> |
| <msgtext> |
| <para> |
| <itemizedlist> |
| <listitem> |
| <para> |
| On Windows 9x (and DOS), |
| <filename>J:\toto\gee\bar.txt</filename>. |
| </para> |
| </listitem> |
| <listitem> |
| <para> |
| On Windows NT, |
| <filename>J:\gee\bar.txt</filename>. |
| </para> |
| </listitem> |
| <listitem> |
| <para> |
| On Windows NT, |
| <filename>J:\tata\titi\bar.txt</filename>. |
| </para> |
| </listitem> |
| </itemizedlist> |
| </para> |
| </msgtext> |
| </entry> |
| <entry> |
| <msgtext> |
| <para> |
| <itemizedlist> |
| <listitem> |
| <para> |
| On Windows NT (and DOS), |
| <filename>\toto</filename> is the default |
| directory on drive <filename>J:</filename>. |
| </para> |
| </listitem> |
| <listitem> |
| <para> |
| On Windows NT, if <envar>=J:</envar> isn't set. |
| </para> |
| </listitem> |
| <listitem> |
| <para> |
| On Windows NT, if <envar>=J:</envar> is set to |
| <filename>J:\tata\titi</filename>. |
| </para> |
| </listitem> |
| </itemizedlist> |
| </para> |
| </msgtext> |
| </entry> |
| </row> |
| <row> |
| <entry>UNC (Uniform Naming Convention) path</entry> |
| <entry><filename>\\host\share\foo\bar.txt</filename></entry> |
| <entry> |
| <filename>\Global??\UNC\host\share\foo\bar.txt</filename> |
| </entry> |
| <entry> |
| Simple concatenation. |
| </entry> |
| </row> |
| <row> |
| <entry>Device path</entry> |
| <entry><filename>\\.\device</filename></entry> |
| <entry><filename>\Global??\device</filename></entry> |
| <entry>Simple concatenation</entry> |
| </row> |
| <row> |
| <entry>Long paths</entry> |
| <entry><filename>\\?\...</filename></entry> |
| <entry></entry> |
| <entry> |
| With this prefix, paths can take up to |
| <constant>32,767</constant> characters, instead of |
| <constant>MAX_PATH</constant> for all the others). Once |
| the prefix stripped, to be handled like one of the |
| previous ones, just providing internal buffers large |
| enough). |
| </entry> |
| </row> |
| </tbody> |
| </tgroup> |
| </table> |
| </para> |
| </sect3> |
| </sect2> |
| |
| <sect2> |
| <title>Wine implementation</title> |
| <para> |
| We'll mainly cover in this section the way Wine opens a file (in the |
| Unix sense) when given a Windows file name. This will include mapping |
| the Windows path onto a Unix path (including the devices case), |
| handling the access rights, the sharing attribute if any... |
| </para> |
| <sect3> |
| <title>Mapping a Windows path into an absolute Windows path</title> |
| <para> |
| First of all, we described in previous section the way to convert |
| any path in an absolute path. Wine implements all the previous algorithms |
| in order to achieve this. Note also, that this transformation is |
| done with information local to the process (default directory, |
| environment variables...). We'll assume in the rest of this section |
| that all paths have now been transformed into absolute from. |
| </para> |
| </sect3> |
| <sect3> |
| <title>Mapping a Windows (absolute) path onto a Unix path</title> |
| <para> |
| When Wine is requested to map a path name (in DOS form, with a drive |
| letter, e.g. <filename>c:\foo\bar\myfile.txt</filename>), Wine |
| converts this into the following Unix path |
| <filename>$(WINEPREFIX)/dosdevices/c:/foo/bar/myfile.txt</filename>. |
| The Wine configuration process is responsible for setting |
| <filename>$(WINEPREFIX)/dosdevices/c:</filename> to be a symbolic |
| link pointing to the directory in Unix hierarchy the user wants to |
| expose as the <filename>C:</filename> drive in the DOS forest of |
| drives. |
| </para> |
| <para> |
| This scheme allows: |
| <itemizedlist> |
| <listitem> |
| <para> |
| a very simple algorithm to map a DOS path name into a Unix one |
| (no need of Wine server calls) |
| </para> |
| </listitem> |
| <listitem> |
| <para> |
| a very configurable implementation: it's very easy to change a |
| drive mapping |
| </para> |
| </listitem> |
| <listitem> |
| <para> |
| a rather readable configuration: no need of sophisticated |
| tools to read a drive mapping, a <command>ls -l |
| $(WINEPREFIX)/dosdevices</command> |
| says it all. |
| </para> |
| </listitem> |
| </itemizedlist> |
| </para> |
| <para> |
| This scheme is also used to implement UNC path names. For example, |
| Wine maps <filename>\\host\share\foo\bar\MyRemoteFile.txt</filename> |
| into |
| <filename>$(WINEPREFIX)/dosdevices/unc/host/share/foo/bar/MyRemoteFile.txt</filename>. |
| It's then up to the user to decide where |
| <filename>$(WINEPREFIX)/dosdevices/unc/host/share</filename> shall |
| point to (or be). For example, it can either be a symbolic link to a |
| directory inside the local machine (just for emulation purpose), or |
| a symbolic link to the mount point of a remote disk (done through |
| Samba or NFS), or even the real mount point. Wine will not do any |
| checking here, nor will help in actually mounting the remote drive. |
| </para> |
| <para> |
| We've seen how Wine maps a drive letter or a UNC path onto the Unix |
| hierarchy, we now have to look on a the filename is searched within |
| this hierarchy. The main issue is about case sensivity. Here's a |
| reminder of the various properties for the file systems in the |
| field. |
| <table> |
| <title>File systems' properties</title> |
| <tgroup cols="4" align="left"> |
| <thead> |
| <row> |
| <entry>FS Name</entry> |
| <entry>Length of elements</entry> |
| <entry>Case sensitivity (on disk)</entry> |
| <entry>Case sensitivity for lookup</entry> |
| </row> |
| </thead> |
| <tbody> |
| <row> |
| <entry>FAT, FAT16 or FAT32</entry> |
| <entry>Short name (8+3)</entry> |
| <entry>Names are always stored in upper-case</entry> |
| <entry>Case insensitive</entry> |
| </row> |
| <row> |
| <entry>VFAT</entry> |
| <entry>Short name (8+3) + alias on long name</entry> |
| <entry> |
| Short names are always stored in upper-case. Long names |
| are stored with case preservation. |
| </entry> |
| <entry>Case insensitive</entry> |
| </row> |
| <row> |
| <entry>NTFS</entry> |
| <entry>Long name + alias on short name (8+3).</entry> |
| <entry> |
| Long names are stored with case preservation. Short names |
| are always stored in upper-case. |
| </entry> |
| <entry>Case insentivite</entry> |
| </row> |
| <row> |
| <entry>Linux FS (ext2fs, ext3fs, reiserfs...)</entry> |
| <entry>Long name</entry> |
| <entry>Case preserving</entry> |
| <entry>Case sensitive</entry> |
| </row> |
| </tbody> |
| </tgroup> |
| </table> |
| </para> |
| <para> |
| <note> |
| <title>Case sensitivity vs. preservation</title> |
| <para> |
| When we say that most systems in NT are case insensitive, this |
| has to be understood for looking up for a file, where the |
| matches are made in a case insensitive mode. This is different |
| from VFAT or NTFS "case preservation" mechanism, which stores |
| the file names as they are given when creating the file, while |
| doing case insensitive matches. |
| </para> |
| </note> |
| Since most file systems used in NT are case insensitive and since |
| most Unix file systems are case sensitive, Wine undergo a case |
| insensitive search when it has found the Unix path is has to look |
| for. This means, for example, that for opening the |
| <filename>$(WINEPREFIX)/dosdevices/c:/foo/bar/myfile.txt</filename>, |
| Wine will recursively open all directories in the path, and check, |
| in this order, for the existence of the directory entry in the form |
| given in the file name (ie. case sensitive), and if it's not found, |
| in a case insensitive form. This allows to also pass, in most Win32 |
| file API also a Unix path (instead of a DOS or NT path), but we'll |
| come back to this later. This also means that the algorithm |
| described doesn't correctly handle the case of two files in the same |
| directory, which names only differ on the case of the letters. This |
| means, that if, in the same directory, two files (which names match |
| in a case sensitive comparison), Wine will pick-up the right one if |
| the filename given matches on of the name (in a case sensitive way), |
| but will pickup one of the two (without defining the one it's going |
| to pickup) if the filename given matches none of the two names in a |
| case sensitive way (but in a case insensitive way). For example, if |
| the two filenames are <filename>my_neat_file.txt</filename> and |
| <filename>My_Neat_File.txt</filename>, Wine's behavior when opening |
| <filename>MY_neat_FILE.txt</filename> is undefined. |
| </para> |
| <para> |
| As Windows, at the early days, didn't support the notion of symbolic |
| links on directories, lots of applications (and some old native |
| DLLs) are not ready for this feature. Mainly, they imply that the |
| directory structure is a tree, which has lots of consequences on |
| navigating in the forest of directories (ie: there cannot be two |
| ways for going from directory to another, there cannot be |
| cycles...). In order to prevent some bad behavior for such |
| applications, Wine sets up an option. By default, symbolic links on |
| directories are not followed by Wine. There's an options to follow |
| them (see the Wine User Guide), but this could be harmful. |
| </para> |
| <para> |
| Wine considers that Unix file names <emphasis>are</emphasis> long |
| filename. This seems a reasonable approach; this is also the |
| approach followed by most of the Unix OSes while mounting Windows |
| partitions (with filesystems like FAT, FAT32 or NTFS). Therefore, |
| Wine tries to support short names the best it can. Basically, they |
| are two options: |
| <itemizedlist> |
| <listitem> |
| <para> |
| The filesystem on which the inspected directory lies in a real |
| Windows FS (like FAT, or FAT32, or NTFS) and the OS has |
| support to access the short filename (for example, Linux does |
| this on FAT, FAT32 or VFAT). In this case, Wine makes full use |
| of this information and really mimics the Windows behavior: |
| the short filename used for any file is the same than on |
| Windows. |
| </para> |
| </listitem> |
| <listitem> |
| <para> |
| If conditions listed above are not met (either, FS has no |
| physical short name support, or OS doesn't provide the access |
| access to the short name), Wine decides and computes on its |
| own the short filename for a given long filename. We cannot |
| ensure that the generated short name is the same than on |
| Windows (because the algorithm on Windows takes into account |
| the order of creation of files, which cannot be implemented in |
| Wine: Wine would have to cache the short names of every |
| directory it uses!). The short name is made up of part of the |
| long name (first characters) and the rest with a hashed |
| value. This has several advantages: |
| <itemizedlist> |
| <listitem> |
| <para> |
| The algorithm is rather simple and low cost. |
| </para> |
| </listitem> |
| <listitem> |
| <para> |
| The algorithm is stateless (doesn't depend of the other |
| files in the directory). |
| </para> |
| </listitem> |
| </itemizedlist> |
| But, it also has the drawbacks (of the advantages): |
| <itemizedlist> |
| <listitem> |
| <para> |
| The algorithm isn't the same as on Windows, which means |
| a program cannot use short names generated on |
| Windows. This could happen when copying an existing |
| installed program from Windows (for example, on a dual |
| boot machine). |
| </para> |
| </listitem> |
| <listitem> |
| <para> |
| Two long file names can end up with the same short name |
| (Windows handles the collision in this case, while Wine |
| doesn't). We rely on our hash algorithm to lower at most |
| this possibility (even if it exists). |
| </para> |
| </listitem> |
| </itemizedlist> |
| </para> |
| </listitem> |
| </itemizedlist> |
| </para> |
| <para> |
| Wine also allows in most file API to give as a parameter a full Unix |
| path name. This is handy when running a Wine (or Winelib) program |
| from the command line, and one doesn't need to convert the path into |
| the Windows form. However, Wine checks that the Unix path given can |
| be accessed from one of the defined drives, insuring that only part |
| of the Unix <filename>/</filename> hierarchy can be accessed. |
| </para> |
| <para> |
| As a side note, as Unix doesn't widely provide a Unicode interface |
| to the filenames, and that Windows implements filenames as Unicode |
| strings (even on the physical layer with NTFS, the FATs variant are |
| ANSI), we need to properly map between the two. At startup, Wine |
| defines what's called the Unix Code Page, that's is the code page |
| the Unix kernel uses as a reference for the strings. Then Wine uses |
| this code page for all the mappings it has to do between a Unicode |
| path (on the Windows side) and a Ansi path to be used in a Unix path |
| API. Note, that this will work as long as a disk isn't mounted with |
| a different code page than the one the kernel uses as a default. |
| </para> |
| <para> |
| We describe below how Windows devices are mapped to Unix devices. |
| Before that, let's finish the pure file round-up with some basic |
| operations. |
| </para> |
| </sect3> |
| <sect3> |
| <title>Access rights and file attributes</title> |
| <para> |
| Now that we have looked how Wine converts a Windows pathname into a |
| Unix one, we need to cover the various meta-data attached to a file |
| or a directory. |
| </para> |
| <para> |
| In Windows, access rights are simplistic: a file can be read-only or |
| read-write. Wine sets the read-only flag if the file doesn't have |
| the Unix user-write flag set. As a matter of fact, there's no way |
| Wine can return that a file cannot be read (that doesn't exist under |
| Windows). The file will be seen, but trying to open it will return |
| an error. The Unix exec-flag is never reported. Wine doesn't use |
| this information to allow/forbid running a new process (as Unix does |
| with the exec-flag). Last but not least: hidden files. This exists |
| on Windows but not really on Unix! To be exact, in Windows, the |
| hidden flag is a metadata associated to any file or directoy; in |
| Unix, it's a convention based on the syntax of the file name |
| (whether it starts with a '.' or not). Wine implements two behaviors |
| (chosen by configuration). This impacts file names and directory |
| names starting by a '.'. In first mode |
| (<option>ShowDotFile</option> is <constant>FALSE</constant>), every |
| file or directory starting by '.' is returned with the hidden flag |
| turned on. This is the natural behavior on Unix (for |
| <command>ls</command> or even file explorer). In the second mode |
| (<option>ShowDotFile</option> is <constant>TRUE</constant>), Wine |
| never sets the hidden flag, hence every file will be seen. |
| </para> |
| <para> |
| Last but not least, before opening a file, Windows makes use of |
| sharing attributes in order to check whether the file can be opened; |
| for example, a process, being the first in the system to open a |
| given file, could forbid, while it maintains the file opened, that |
| another process opens it for write access, whereas open for read |
| access would be granted. This is fully supported in Wine by moving |
| all those checks in the Wine server for a global view on the system. |
| Note also that what's moved in the Wine server is the check, when |
| the file is opened, to implement the Windows sharing semantics. |
| Further operation on the file (like reading and writing) will not |
| require heavy support from the server. |
| </para> |
| <para> |
| The other good reason for putting the code for actually opening a |
| file in the server is that an opened files in Windows is managed |
| through a handle, and handles can only be created in Wine server! |
| </para> |
| <para> |
| Just a note about attributes on directories: while we can easily map |
| the meaning of Windows' <constant>FILE_ATTRIBUTE_READONLY</constant> |
| on a file, we cannot do it for a directory. Windows' semantic (when |
| this flag is set) means do not delete the directory, while the |
| <constant>w</constant> attribute in Unix means don't write nor |
| delete it. Therefore, Wine uses an asymetric mapping here: if the |
| directory (in Unix) isn't writable, then Wine reports the |
| <constant>FILE_ATTRIBUTE_READONLY</constant> attribute; on the other |
| way around, when asked to set a directory with |
| <constant>FILE_ATTRIBUTE_READONLY</constant> attribute, Wine simply |
| does nothing. |
| </para> |
| </sect3> |
| <sect3> |
| <title>Operations on file</title> |
| <sect4> |
| <title>Reading and writing</title> |
| <para> |
| Reading and writing are the basic operations on files. Wine of |
| course implements this, and bases the implementation on client |
| side calls to Unix equivalents (like <function>read()</function> |
| or <function>write()</function>). Note, that the Wine server is |
| involved in any read or write operation, as Wine needs to |
| transform the Windows-handle to the file into a Unix file |
| descriptor it can pass to any Unix file function. |
| </para> |
| </sect4> |
| <sect4> |
| <title>Getting a Unix fd</title> |
| <para> |
| This is major operation in any file related operation. Basically, |
| each file opened (at the Windows level), is first opened in the |
| Wine server, where the fd is stored. Then, Wine (on client side) |
| uses <function>recvmsg()</function> to pass the fd from the wine |
| server process to the client process. Since this operation could |
| be lengthy, Wine implement some kind of cache mechanism to send it |
| only once, but getting a fd from a handle on a file (or any other |
| Unix object which can be manipulated through a file descriptor) |
| still requires a round trip to the Wine server. |
| </para> |
| </sect4> |
| <sect4> |
| <title>Locking</title> |
| <para> |
| Windows provides file locking capabilities. When a lock is set |
| (and a lock can be set on any contiguous range in a file), it |
| controls how other processes in the system will have access to the |
| range in the file. Since locking range on a file are defined on a |
| system wide manner, its implementation resides in |
| <command>wineserver</command>. It tries to make use Unix file |
| locking (if the underlying OS and the mounted disk where the file |
| sits support this feature) with <function>fcntl()</function> and |
| the <constant>F_SETLK</constant> command. If this isn't |
| supported, then <command>wineserver</command> just pretends it |
| works. |
| </para> |
| </sect4> |
| <sect4> |
| <title>I/O control</title> |
| <para> |
| There's no need (so far) to implement support (for files and |
| directories) for <function>DeviceIoControl()</function>, even if |
| this is supported by Windows, but for very specific needs |
| (like compression management, or file system related information). |
| This isn't the case for devices (including disks), but we'll cover |
| this in the hereafter section related to devices. |
| </para> |
| </sect4> |
| <sect4> |
| <title>Buffering</title> |
| <para> |
| Wine doesn't do any buffering on file accesses but rely on the |
| underlying Unix kernel for that (when possible). This scheme is |
| needed because it's easier to implement multiple accesses on the |
| same file at the kernel level, rather than at Wine levels. Doing |
| lots of small reads on the same file can turn into a performance |
| hog, because each read operation needs a round trip to the server |
| in order to get a file descriptor (see above). |
| </para> |
| </sect4> |
| <sect4> |
| <title>Overlapped I/O</title> |
| <para> |
| Windows introduced the notion of overlapped I/O. Basically, it |
| just means that an I/O operation (think read / write to start |
| with) will not wait until it's completed, but rather return to the |
| caller as soon as possible, and let the caller handle the wait |
| operation and determine when the data is ready (for a read |
| operation) or has been sent (for a write operation). Note that the |
| overlapped operation is linked to a specific thread. |
| </para> |
| <para> |
| There are several interests to this: a server can handle several |
| clients without requiring multi-threading techniques; you can |
| handle an event driven model more easily (ie how to kill properly |
| a server while waiting in the lengthy <function>read()</function> |
| operation). |
| </para> |
| <para> |
| Note that Microsoft's support for this feature evolved along the |
| various versions of Windows. For example, Windows 95 or 98 only |
| supports overlapped I/O for serial and parallel ports, while NT |
| supports also files, disks, sockets, pipes, or mailslots. |
| </para> |
| <para> |
| Wine implements overlapped I/O operations. This is mainly done by |
| queueing in the server a request that will be triggered when |
| something the current state changes (like data available for a |
| read operation). This readiness is signaled to the calling |
| processing by queueing a specific APC, which will be called within |
| the next waiting operation the thread will have. This specific |
| APC will then do the hard work of the I/O operation. This scheme |
| allows to put in place a wait mechanism, to attach a routine to be |
| called (on the thread context) when the state changes, and to be |
| done is a rather transparent manner (embedded any the generic wait |
| operation). However, it isn't 100% perfect. As the heavy |
| operations are done in the context of the calling threads, if |
| those operations are lengthy, there will be an impact on the |
| calling thread, especially its latency. In order to provide an |
| effective support for this overlapped I/O operations, we would |
| need to rely on Unix kernel features (AIO is a good example). |
| </para> |
| </sect4> |
| </sect3> |
| <sect3> |
| <title>Devices & volume management</title> |
| <para> |
| We've covered so far the ways file names are mapped into Unix |
| paths. There's still need to cover it for devices. As a regular |
| file, devices are manipulated in Windows with both read / write |
| operations, but also control mechanisms (speed or parity of a serial |
| line; volume name of a hard disk...). Since, this is also supported |
| in Linux, there's also a need to open (in a Unix sense) a device |
| when given a Windows device name. This section applies to DOS device |
| names, which are seen in NT as nicknames to other devices. |
| </para> |
| <para> |
| Firstly, Wine implements the Win32 to NT mapping as described above, |
| hence every device path (in NT sense) is of the following form: |
| <filename>/??/devicename</filename> (or |
| <filename>/DosDevices/devicename</filename>). As Windows device |
| names are case insensitive, Wine also converts them to lower case |
| before any operation. Then, the first operation Wine tries is to |
| check whether |
| <filename>$(WINEPREFIX)/dosdevices/devicename</filename> exists. If |
| so, it's used as the final Unix path for the device. The |
| configuration process is in charge of creating for example, a |
| symbolic link between |
| <filename>$(WINEPREFIX)/dosdevices/PhysicalDrive0</filename> and |
| <filename>/dev/hda0</filename>. If such a link cannot be found, and |
| the device name looks like a DOS disk name (like |
| <filename>C:</filename>), Wine first tries to get the Unix device |
| from the path <filename>$(WINEPREFIX)/dosdevices/c:</filename> |
| (i.e. the device which is mounted on the target of the symbol link); |
| if this doesn't give a Unix device, Wine tries whether |
| <filename>$(WINEPREFIX)/dosdevices/c::</filename> exists. If so, |
| it's assumed to be a link to the actual Unix device. For example, |
| for a CD Rom, <filename>$(WINEPREFIX)/dosdevices/e::</filename> |
| would be a symbolic link to <filename>/dev/cdrom</filename>. If |
| this doesn't exist (we're still handling the a device name of the |
| <filename>C:</filename> form), Wine tries to get the Unix device |
| from the system information (<filename>/etc/mtab</filename> and |
| <filename>/etc/fstab</filename> on Linux). We cannot apply this |
| method in all the cases, because we have no insurance that the |
| directory can actually be found. One could have, for example, a CD |
| Rom which he/she want only to use as audio CD player (ie never |
| mounted), thus not having any information of the device itself. If |
| all of this doesn't work either, some basic operations are checked: |
| if the devicename is <filename>NUL</filename>, then |
| <filename>/dev/null</filename> is returned. If the device name is a |
| default serial name (<filename>COM1</filename> up to |
| <filename>COM9</filename>) (resp. printer name |
| <filename>LPT1</filename> up to <filename>LPT9</filename>), then |
| Wine tries to open the Nth serial (resp. printer) in the system. |
| Otherwise, some basic old DOS name support is done |
| <filename>AUX</filename> is transformed into |
| <filename>COM1</filename> and <filename>PRN</filename> into |
| <filename>LPT1</filename>), and the whole process is retried with |
| those new names. |
| </para> |
| <para> |
| To sum up: |
| <table> |
| <title> |
| Mapping of Windows device names into Unix device names |
| </title> |
| <tgroup cols="3" align="left"> |
| <thead> |
| <row> |
| <entry>Windows device name</entry> |
| <entry>NT device name</entry> |
| <entry>Mapping to Unix device name</entry> |
| </row> |
| </thead> |
| <tbody> |
| <row> |
| <entry><filename><any_path>AUX</filename></entry> |
| <entry<filename>>\Global??\AUX</filename></entry> |
| <entry> |
| Treated as an alias to <filename>COM1</filename> |
| </entry> |
| </row> |
| <row> |
| <entry><filename><any_path>PRN</filename></entry> |
| <entry><filename>\Global??\PRN</filename></entry> |
| <entry>Treated as an alias to <filename>LPT1</filename></entry> |
| </row> |
| <row> |
| <entry><filename><any_path>COM1</filename></entry> |
| <entry><filename>\Global??\COM1</filename></entry> |
| <entry> |
| <filename>$(WINEPREFIX)/dosdevices/com1</filename> |
| (if the symbol link exists) or the Nth serial |
| line in the system (on Linux, |
| <filename>/dev/ttyS0</filename>). |
| </entry> |
| </row> |
| <row> |
| <entry><filename><any_path>LPT1</filename></entry> |
| <entry><filename>\Global??\LPT1</filename></entry> |
| <entry> |
| <filename>$(WINEPREFIX)/dosdevices/lpt1</filename> |
| (if the symbol link exists) or the Nth printer |
| in the system (on Linux, |
| <filename>/dev/lp0</filename>). |
| </entry> |
| </row> |
| <row> |
| <entry><filename><any_path>NUL</filename></entry> |
| <entry><filename>\Global??\NUL</filename></entry> |
| <entry><filename>/dev/null</filename></entry> |
| </row> |
| <row> |
| <entry><filename>\\.\E:</filename></entry> |
| <entry><filename>\Global??\E:</filename></entry> |
| <entry> |
| <filename>$(WINEPREFIX)/dosdevices/e::</filename> (if the |
| symbolic link exists) or guessing the device from |
| <filename>/etc/mtab</filename> or |
| <filename>/etc/fstab</filename>. |
| </entry> |
| </row> |
| <row> |
| <entry><filename>\\.\<device_name></filename></entry> |
| <entry> |
| <filename>\Global??\<device_name></filename> |
| </entry> |
| <entry> |
| <filename>$(WINEPREFIX)/dosdevices/<device_name></filename> |
| (if the symbol link exists). |
| </entry> |
| </row> |
| </tbody> |
| </tgroup> |
| </table> |
| </para> |
| <para> |
| Now that we know which Unix device to open for a given Windows |
| device, let's cover the operation on it. Those operations can either |
| be read / write, io control (and even others). |
| </para> |
| <para> |
| Read and write operations are supported on Real disks & CDROM |
| devices, under several conditions: |
| <itemizedlist> |
| <listitem> |
| <para> |
| Foremost, as the <function>ReadFile()</function> and |
| <function>WriteFile()</function> calls are mapped onto the |
| Unix <function>read()</function> and |
| <function>write()</function> calls, the user (from the Unix |
| perspective of the one running the Wine executable) must have |
| read (resp. write) access to the device. It wouldn't be wise |
| to let a user write directly to a hard disk!!! |
| </para> |
| </listitem> |
| <listitem> |
| <para> |
| Blocks' size for read and write but be of the size of a |
| physical block (generally 512 for a hard disk, depends on the |
| type of CD used), and offsets must also be a multiple of the |
| block size. |
| </para> |
| </listitem> |
| </itemizedlist> |
| </para> |
| <para> |
| Wine also reads (if the first condition above about access rights is |
| met) the volume information from a hard disk or a CD ROM to be |
| displayed to a user. |
| </para> |
| <!-- |
| <para> |
| Handling of old DOS devices (<filename>COMx</filename>, |
| <filename>LPTx</filename>, <filename>NUL</filename>...) |
| </para> |
| --> |
| <para> |
| Wine also recognizes VxD as devices. But those VxD must be the |
| Wine builtin ones (Wine will never allow to load native VxD). Those |
| are configured with symbolic links in the |
| <filename>$(WINEPREFIX)/dosdevices/</filename> directory, and point |
| to the actual builtin DLL. This DLL exports a single entry point, |
| that Wine will use when a call to |
| <function>DeviceIoControl</function> is made, with a handle opened |
| to this VxD. This allows to provide some kind of compatibility for |
| old Win9x apps, still talking directly to VxD. This is no longer |
| supported on Windows NT, newest programs are less likely to make use |
| of this feature, so we don't expect lots of development in this |
| area, even though the framework is there and working. Note also that |
| Wine doesn't provide support for native VxDs (as a game, report how |
| many times this information is written in the documentation; as an |
| advanced exercise, find how many more occurrences we need in order to |
| stop questions whether it's possible or not). |
| </para> |
| </sect3> |
| </sect2> |
| </sect1> |
| <sect1 id="ntdll"> |
| <title><filename>NTDLL</filename> module</title> |
| <para> |
| <filename>NTDLL</filename> provides most of the services you'd expect |
| from a kernel. In lots of cases, <filename>KERNEL32</filename> APIs are |
| just wrappers to <filename>NTDLL</filename> APIs. There are however, |
| some difference in the APIs (the <filename>NTDLL</filename> ones have |
| quite often a bit wider semantics than their |
| <filename>KERNEL32</filename> counterparts). All the detailed functions |
| we've described since the beginning of this chapter are in fact |
| implemented in <filename>NTDLL</filename>, plus a great numbers of |
| others we haven's written about yet. |
| </para> |
| </sect1> |
| |
| <sect1> |
| <title><filename>KERNEL32</filename> Module</title> |
| |
| <para> |
| As already explained, <filename>KERNEL32</filename> maps quite a few of |
| its APIs to <filename>NTDLL</filename>. There are however a couple of |
| things which are handled directly in |
| <filename>KERNEL32</filename>. Let's cover a few of them... |
| </para> |
| <sect2 id="consoles"> |
| <title>Console</title> |
| <sect3> |
| <title>NT implementation</title> |
| <para> |
| Windows implements console solely in the Win32 subsystem. Under NT, |
| the real implementation uses a dedicated subsystem |
| <filename>csrss.exe</filename> Client/Server Run-time SubSystem) |
| which is in charge, amont other things, of animating the consoles. |
| Animating includes for example handling several processes on the |
| same console (write operations must be atomic, but also a character |
| keyed on the console must be read by a single process), or sending |
| some information back to the processes (changing the size or |
| attributes of the console, closing the console). Windows NT uses a |
| dedicated (RPC based) protocol between each process being attached |
| to a console and the <command>csrss.exe</command> subsystem, which |
| is in charge of the UI of every console in the system. |
| </para> |
| </sect3> |
| <sect3> |
| <title>Wine implementation</title> |
| <para> |
| Wine tries to integrate as much as possible into the Unix consoles, |
| but the overall situation isn't perfect yet. Basically, Wine |
| implements three kinds of consoles: |
| <itemizedlist> |
| <listitem> |
| <para> |
| the first one is a direct mapping of the Unix console into the |
| Windows environment. From the windows program point of view, |
| it won't run in a Windows console, but it will see its |
| standard input and output streams redirected to files; thoses |
| files are hooked into the Unix console's output and input |
| streams respectively. This is handy for running programs from |
| a Unix command line (and use the result of the program as it |
| was a Unix programs), but it lacks all the semantics of the |
| Windows consoles. |
| </para> |
| </listitem> |
| <listitem> |
| <para> |
| the second and third ones are closer to the NT scheme, albeit |
| different from what NT does. The <command>wineserver</command> |
| plays the role of the <filename>csrss.exe</filename> subsystem |
| (all requests are sent to it), and are then dispatched to a |
| dedicated wine process, called (surprise!) |
| <command>wineconsole</command> which manages the UI of the |
| console. There is a running instance of |
| <command>wineconsole</command> for every console in the |
| system. Two flavors of this scheme are actually implemented: |
| they vary on the backend for the |
| <command>wineconsole</command>. The first one, dubbed |
| <constant>user</constant>, creates a real GUI window |
| (hence the USER name) and renders the console in this window. |
| The second one uses the <filename>(n)curses</filename> library |
| to take full control of an existing Unix console; of course, |
| interaction with other Unix programs will not be as smooth as |
| the first solution. |
| </para> |
| </listitem> |
| </itemizedlist> |
| </para> |
| <para> |
| The following table describes the main implementation differences |
| between the three approaches. |
| <table> |
| <title>Function consoles implementation comparison</title> |
| <tgroup cols="4" align="left"> |
| <thead> |
| <row> |
| <entry>Function</entry> |
| <entry>Bare streams</entry> |
| <entry> |
| <command>Wineconsole</command> & user backend |
| </entry> |
| <entry> |
| <command>Wineconsole</command> & curses backend |
| </entry> |
| </row> |
| </thead> |
| <tbody> |
| <row> |
| <entry> |
| Console as a Win32 Object (and associated handles) |
| </entry> |
| <entry> |
| No specific Win32 object is used in this case. The |
| handles manipulated for the standard Win32 streams are in |
| fact "bare handles" to their corresponding Unix streams. |
| The mode manipulation functions |
| (<function>GetConsoleMode()</function> / |
| <function>SetConsoleMode()</function>) are not supported. |
| </entry> |
| <entry> |
| Implemented in server, and a specific Winelib program |
| (<command>wineconsole</command>) is in charge of the |
| rendering and user input. The mode manipulation functions |
| behave as expected. |
| </entry> |
| <entry> |
| Implemented in server, and a specific Winelib program |
| (<command>wineconsole</command>) is in charge of the |
| rendering and user input. The mode manipulation functions |
| behave as expected. |
| </entry> |
| </row> |
| <row> |
| <entry> |
| Inheritance (including handling in |
| <function>CreateProcess()</function> of |
| <constant>CREATE_DETACHED</constant>, |
| <constant>CREATE_NEW_CONSOLE</constant> flags). |
| </entry> |
| <entry> |
| Not supported. Every process child of a process will |
| inherit the Unix streams, so will also inherit the Win32 |
| standard streams. |
| </entry> |
| <entry> |
| Fully supported (each new console creation will be handled |
| by the creation of a new <filename>USER32</filename> window) |
| </entry> |
| <entry> |
| Fully supported, except for the creation of a new console, |
| which will be rendered on the same Unix terminal as the |
| previous one, leading to unpredictable results. |
| </entry> |
| </row> |
| <row> |
| <entry> |
| <function>ReadFile()</function> / |
| <function>WriteFile()</function> operations |
| </entry> |
| <entry>Fully supported</entry> |
| <entry>Fully supported</entry> |
| <entry>Fully supported</entry> |
| </row> |
| <row> |
| <entry> |
| Screen-buffer manipulation (creation, deletion, resizing...) |
| </entry> |
| <entry>Not supported</entry> |
| <entry>Fully supported</entry> |
| <entry> |
| Partly supported (this won't work too well as we don't |
| control (so far) the size of underlying Unix terminal |
| </entry> |
| </row> |
| <row> |
| <entry> |
| APIs for reading/writing screen-buffer content, cursor position |
| </entry> |
| <entry>Not supported</entry> |
| <entry>Fully supported</entry> |
| <entry>Fully supported</entry> |
| </row> |
| <row> |
| <entry>APIs for manipulating the rendering window size</entry> |
| <entry>Not supported</entry> |
| <entry>Fully supported</entry> |
| <entry> |
| Partly supported (this won't work too well as we don't |
| control (so far) the size of underlying Unix terminal |
| </entry> |
| </row> |
| <row> |
| <entry> |
| Signaling (in particular, Ctrl-C handling) |
| </entry> |
| <entry> |
| Nothing is done, which means that Ctrl-C will generate (as |
| usual) a <constant>SIGINT</constant> which will terminate |
| the program. |
| </entry> |
| <entry> |
| Partly supported (Ctrl-C behaves as expected, however the |
| other Win32 CUI signaling isn't properly implemented). |
| </entry> |
| <entry> |
| Partly supported (Ctrl-C behaves as expected, however the |
| other Win32 CUI signaling isn't properly implemented). |
| </entry> |
| </row> |
| </tbody> |
| </tgroup> |
| </table> |
| </para> |
| <para> |
| The Win32 objects behind a console can be created in several |
| occasions: |
| <itemizedlist> |
| <listitem> |
| <para> |
| When the program is started from |
| <command>wineconsole</command>, a new console object is |
| created and will be used (inherited) by the process launched |
| from <command>wineconsole</command>. |
| </para> |
| </listitem> |
| <listitem> |
| <para> |
| When a program, which isn't attached to a console, calls |
| <function>AllocConsole()</function>, Wine then launches |
| <command>wineconsole</command>, and attaches the current |
| program to this console. In this mode, the |
| <filename>USER32</filename> mode is always selected as Wine |
| cannot tell the current state of the Unix console. |
| </para> |
| </listitem> |
| </itemizedlist> |
| </para> |
| <para> |
| Please also note, that starting a child process with the |
| <constant>CREATE_NEW_CONSOLE</constant> flag, will end-up calling |
| <function>AllocConsole()</function> in the child process, hence |
| creating a <command>wineconsole</command> with the |
| <filename>USER32</filename> backend. |
| </para> |
| <para> |
| Another interesting point to note is that Windows implements handles |
| to console objects (input and screen buffers) only in the |
| <filename>KERNEL32</filename> DLL, and those are not sent nor seen |
| from the <filename>NTDLL</filename> level, albeit, for example, |
| console are waitable on input. How is this possible? Well, Windows |
| NT is a bit tricky here. Regular handles have an interesting |
| property: their integral value is always a multiple of four (they |
| are likely to be offsets from the beginning of a table). Console |
| handles, on the other hand, are not multiple of four, but have the |
| two lower bit set (being a multiple of four means having the two |
| lower bits reset). When <filename>KERNEL32</filename> sees a handle |
| with the two lower bits set, it then knows it's a console handle and |
| takes appropriate decisions. For example, in the various |
| <function>kernel32!WaitFor*()</function> functions, it transforms |
| any console handle (input and <emphasis>output</emphasis> - |
| strangely enough handles to console's screen buffers are waitable) |
| into a dedicated wait event for the targetted console. There's an |
| (undocumented) <filename>KERNEL32</filename> function |
| <function>GetConsoleInputWaitHandle()</function> which returns the |
| handle to this event in case you need it. Another interesting |
| handling of those console's handles is in |
| <function>ReadFile()</function> |
| (resp. <function>WriteFile()</function>), which behavior, for |
| console's handles, is transferred to |
| <function>ReadConsole()</function> (resp. |
| <function>WriteConsole()</function>). Note that's always the ANSI |
| version of |
| <function>ReadConsole()</function> / |
| <function>WriteConsole()</function> |
| which is called, hence using the default console's code page. There |
| are some other spots affected, but you can look in |
| <filename>dlls/kernel</filename> to find them all. All of this is |
| implemented in Wine. |
| </para> |
| <para> |
| Wine also implements the same layout of the registry for storing the |
| preferences of the console as Windows does. Those settings can |
| either be defined globally, or on a per process name basis. |
| <command>wineconsole</command> provides the choice to the user to |
| pick you which registry part (global, current running program) it |
| wishes to modify the settings for. |
| <table> |
| <title>Console registry settings</title> |
| <tgroup cols="3" align="left"> |
| <thead> |
| <row> |
| <entry>Name</entry> |
| <entry>Default value</entry> |
| <entry>Purpose</entry> |
| </row> |
| </thead> |
| <tbody> |
| <row> |
| <entry>CursorSize</entry> |
| <entry>25</entry> |
| <entry> |
| Percentage of cell height to which the cursor extents |
| </entry> |
| </row> |
| <row> |
| <entry>CursorVisible</entry> |
| <entry>1</entry> |
| <entry>Whether the cursor is visible or not</entry> |
| </row> |
| <row> |
| <entry>EditionMode</entry> |
| <entry>0</entry> |
| <entry> |
| The way the edition takes place in the console: 0 is |
| insertion mode, 1 is overwrite mode. |
| </entry> |
| </row> |
| <row> |
| <entry>ExitOnDie</entry> |
| <entry>1</entry> |
| <entry> |
| Whether the console should close itself when last running |
| program attached to it dies |
| </entry> |
| </row> |
| <row> |
| <entry>FaceName</entry> |
| <entry>No default</entry> |
| <entry> |
| Name of the font to be used for display. When none is |
| given, <command>wineconsole</command> tries its best to |
| pick up a decent font |
| </entry> |
| </row> |
| <row> |
| <entry>FontSize</entry> |
| <entry>0x0C08</entry> |
| <entry> |
| The high word in the font's cell height, and the low word |
| is the font cell's width. The default value is 12 pixels |
| in height and 8 pixels in width. |
| </entry> |
| </row> |
| <row> |
| <entry>FontWeight</entry> |
| <entry>0</entry> |
| <entry> |
| Weigth of the font. If none is given (or 0) |
| <command>wineconsole</command> picks up a decent font size |
| </entry> |
| </row> |
| <row> |
| <entry>HistoryBufferSize</entry> |
| <entry>50</entry> |
| <entry> |
| Number of entries in history buffer (not actually used) |
| </entry> |
| </row> |
| <row> |
| <entry>HistoryNoDup</entry> |
| <entry>0</entry> |
| <entry> |
| Whether the history should store twice the same entry |
| </entry> |
| </row> |
| <row> |
| <entry>MenuMask</entry> |
| <entry>0</entry> |
| <entry> |
| This mask only exists for Wine console handling. It |
| allows to know which combination of extra keys are need to |
| open the configuration window on right click. The mask |
| can include <constant>MK_CONTROL</constant> or |
| <constant>MK_SHIFT</constant> bits. This can be needed |
| when programs actually need the right click to be passed |
| to them instead of being intercepted by |
| <command>wineconsole</command>. |
| </entry> |
| </row> |
| <row> |
| <entry>QuickEdit</entry> |
| <entry>0</entry> |
| <entry> |
| If null, mouse events are sent to the application. If non |
| null, mouse events are used to select text on the window. |
| This setting must really be set on a application per |
| application basis, because it deals with the fact the CUI |
| application will use or not the mouse events. |
| </entry> |
| </row> |
| <row> |
| <entry>ScreenBufferSize</entry> |
| <entry>0x1950</entry> |
| <entry> |
| The high word is the number of font cells in the height of |
| the screen buffer, while the low word is the number of |
| font cells in the width of the screen buffer. |
| </entry> |
| </row> |
| <row> |
| <entry>ScreenColors</entry> |
| <entry>0x000F</entry> |
| <entry> |
| Default color attribute for the screen buffer (low char is |
| the foreground color, and high char is the background |
| color) |
| </entry> |
| </row> |
| <row> |
| <entry>WindowSize</entry> |
| <entry>0x1950</entry> |
| <entry> |
| The high word is the number of font cells in the height of |
| the window, while the low word is the number of font cells |
| in the width of the window. This window is the visible |
| part of the screen buffer: this implies that a screen |
| buffer must always be bigger than its window, and that the |
| screen buffer can be scrolled so that every cell of the |
| screen buffer can be seen in the window. |
| </entry> |
| </row> |
| </tbody> |
| </tgroup> |
| </table> |
| </para> |
| </sect3> |
| </sect2> |
| <sect2> |
| <title>Win16 processes support</title> |
| <sect3> |
| <title>Starting a NE (Win16) process</title> |
| <para> |
| Wine is also able to run 16 bit processes, but this feature is only |
| supported on Intel IA-32 architectures. |
| </para> |
| <para> |
| When Wine is requested to run a NE (Win 16 process), it will in fact |
| hand over the execution of it to a specific executable |
| <command>winevdm</command>. VDM stands for Virtual DOS |
| Machine. This <command>winevdm</command> is a Winelib application, |
| but will in fact set up the correct 16 bit environment to run the |
| executable. We will get back later on in details to what this means. |
| </para> |
| <para> |
| Any new 16 bit process created by this executable (or its children) |
| will run into the same <command>winevdm</command> instance. Among |
| one instance, several functionalities will be provided to those 16 |
| bit processes, including the cooperative multitasking, sharing the |
| same address space, managing the selectors for the 16 bit segments |
| needed for code, data and stack. |
| </para> |
| <para> |
| Note that several <command>winevdm</command> instances can run in |
| the same Wine session, but the functionalities described above are |
| only shared among a given instance, not among all the |
| instances. <command>winevdm</command> is built as Winelib |
| application, and hence has access to any facility a 32 bit |
| application has. |
| </para> |
| <para> |
| Each Win16 application is implemented in |
| <command>winevdm</command> as a Win32 |
| thread. <command>winevdm</command> then implements its own |
| scheduling facilities (in fact, the code for this feature is in the |
| <filename>krnl386.exe</filename> DLL). Since the required Win16 |
| scheduling is non pre-emptive, this doesn't require any underlying |
| OS kernel support. |
| </para> |
| </sect3> |
| <sect3> |
| <title>SysLevels</title> |
| <para> |
| SysLevels are an undocumented Windows-internal thread-safety system |
| dedicated to 16 bit applications (or 32 bit applications that call - |
| directly or indirectly - 16 bit code). They are basically critical |
| sections which must be taken in a particular order. The mechanism is |
| generic but there are always three syslevels: |
| <itemizedlist> |
| <listitem> |
| <para>level 1 is the Win16 mutex,</para> |
| </listitem> |
| <listitem> |
| <para>level 2 is the <filename>USER</filename> mutex,</para> |
| </listitem> |
| <listitem> |
| <para>level 3 is the <filename>GDI</filename> mutex.</para> |
| </listitem> |
| </itemizedlist> |
| </para> |
| |
| <para> |
| When entering a syslevel, the code (in |
| <filename>dlls/kernel/syslevel.c</filename>) will check that a |
| higher syslevel is not already held and produce an error if so. This |
| is because it's not legal to enter level 2 while holding level 3 - |
| first, you must leave level 3. |
| </para> |
| |
| <para> |
| Throughout the code you may see calls to |
| <function>_ConfirmSysLevel()</function> and |
| <function>_CheckNotSysLevel()</function>. These functions are |
| essentially assertions about the syslevel states and can be used to |
| check that the rules have not been accidentally violated. In |
| particular, <function>_CheckNotSysLevel()</function> will break |
| probably into the debugger) if the check fails. If this happens the |
| solution is to get a backtrace and find out, by reading the source |
| of the wine functions called along the way, how Wine got into the |
| invalid state. |
| </para> |
| </sect3> |
| <sect3> |
| <title>Memory management</title> |
| <para> |
| Every Win16 address is expressed in the form of selector:offset. The |
| selector is an entry in the LDT, but a 16 bit entry, limiting each |
| offset to 64 KB. Hence, the maximum available memory to a Win16 |
| process is 512 MB. Note, that the processor runs in protected mode, |
| but using 16 bit selectors. |
| </para> |
| <para> |
| Windows, for a 16 bit process, defines a few selectors to access the |
| "real" memory (the one provided) by DOS. Basically, Wine also |
| provides this area of memory. |
| </para> |
| </sect3> |
| </sect2> |
| <sect2> |
| <title>DOS processes support</title> |
| <para> |
| The behaviour we just described also applies to DOS executables, which |
| are handled the same way by <command>winevdm</command>. This is only |
| supported on Intel IA-32 architectures. |
| </para> |
| <para> |
| Wine implements also most of the DOS support in a Wine specific DLL |
| (<filename>winedos</filename>). This DLL is called under certain |
| conditions, like: |
| <itemizedlist> |
| <listitem> |
| <para> |
| In <command>winevdm</command>, when trying to launch a DOS |
| application (<filename>.EXE</filename> or |
| <filename>.COM</filename>, <filename>.PIF</filename>). |
| </para> |
| </listitem> |
| <listitem> |
| <para> |
| In <filename>kernel32</filename>, when an attempt is made in the |
| binary code to call some DOS or BIOS interrupts (like Int 21h |
| for example). |
| </para> |
| </listitem> |
| </itemizedlist> |
| </para> |
| <para> |
| When <command>winevdm</command> runs a DOS program, this one runs in |
| real mode (in fact in V86 mode from the IA-32 point of view). |
| </para> |
| <para> |
| Wine also supports part of the DPMI (DOS Protected Mode Interface). |
| </para> |
| <para> |
| Wine, when running a DOS programs, needs to map the 1 MB of virtual |
| memory to the real memory (as seen by the DOS program). When this is |
| not possible (like when someone else is already using this area), the |
| DOS support is not possible. Not also that by doing so, access to |
| linear address 0 is enabled (as it's also real mode address 0 which is |
| valid). Hence, NULL pointer derefence faults are no longer catched. |
| </para> |
| </sect2> |
| </sect1> |
| |
| </chapter> |
| |
| <!-- Keep this comment at the end of the file |
| Local variables: |
| mode: sgml |
| sgml-parent-document:("wine-devel.sgml" "set" "book" "part" "chapter" "") |
| End: |
| --> |