How Windows 98 Managed Network File Locking

Microsoft Windows 98 handled file locking over local area networks primarily through the Server Message Block (SMB) protocol, executed by the operating system's 32-bit network redirector architecture. Operating in mixed environments alongside Windows NT and MS-DOS peers, Windows 98 relied on a combination of file-sharing open modes, byte-range locking, and opportunistic locking (oplocks) to maintain data integrity when multiple clients accessed shared files on network volumes.

The Redirector and Server Architecture

Network I/O in Windows 98 passed through the Installable File System (IFS) Manager, which routed local requests to the file system driver (such as FAT32) and remote requests to the 32-bit Client for Microsoft Networks via the Virtual Redirector driver (VREDIR.VXD). When acting as a file provider through "File and Printer Sharing for Microsoft Networks," the operating system utilized the server driver (VSERVER.VXD).

Communication between nodes took place using SMB over NetBIOS, supported by transport protocols such as NetBEUI, IPX/SPX, or TCP/IP (NetBT). Every file access request, lock grant, and unlock release was translated into discrete SMB commands negotiated between the redirector and the remote file server.

File Share Modes and Deny Flags

Windows 98 supported legacy MS-DOS sharing conventions as well as Win32 API calls. When an application opened a network file via the Win32 CreateFile API or legacy INT 21h Function 3Dh, it defined both access rights and sharing restrictions:

  • Compatibility Mode: Maintained for older 16-bit software, assuming exclusive access unless explicitly shared.
  • Deny Modes: Allowed processes to open files while dictating whether concurrent processes could access them:
    • Deny None: Enabled simultaneous read and write actions by other users.
    • Deny Read: Prevented other network clients from reading the file.
    • Deny Write: Allowed concurrent reads while preventing write access.
    • Deny All (Exclusive): Blocked all other processes from opening the file until the handle was closed.

The server maintained these states in memory tables, rejecting incoming network open requests that conflicted with previously granted share modes.

Byte-Range Locking

For multi-user applications such as database engines (e.g., Microsoft Access and FoxPro), entire-file locking was too restrictive. Windows 98 supported byte-range locking, enabling clients to lock specific segments or records within an open file.

This was handled via the Win32 LockFile and LockFileEx functions, which generated SMB LOCKING_ANDX protocol requests. When a client secured a range of bytes, other clients attempting to read or write to that specific offset received an access violation error (ERROR_LOCK_VIOLATION). Once the transaction completed, the client issued an unlock operation (UnlockFile), freeing the byte range for other networked peers without closing the underlying file handle.

Opportunistic Locking (Oplocks)

To reduce network traffic, Windows 98 supported SMB Opportunistic Locks (Oplocks). An oplock was a guarantee from the server that a client could cache data locally (both read ahead and write behind) without coordinating every I/O transaction over the network wire:

  • Exclusive (Level 1) Oplocks: Granted when only one client had the file open. The client could cache reads and writes locally.
  • Level 2 Oplocks: Allowed multiple clients to cache reads, provided no write operations occurred.
  • Oplock Breaks: If a secondary client requested access to a file held under an exclusive oplock, the server notified the first client with an "oplock break" signal. The first client paused local operations, flushed cached writes across the network, released its lock, and downgraded to a shared lock or normal non-cached state.

Peer-to-Peer Limitations on FAT Systems

When Windows 98 acted as the file host rather than accessing a dedicated Windows NT server, its file locking capabilities were limited by its underlying architecture. The native FAT16 and FAT32 file systems did not provide native object-level access controls or persistent transaction logging.

Lock tracking in VSERVER.VXD occurred entirely in volatile system memory. If a client machine crashed or experienced an unexpected network drop, Windows 98 often failed to detect the severed connection immediately. This caused "orphan locks" or stale lock states, which frequently required a reboot of the host machine to clear the allocated memory tables and make the files accessible again.