ThothII/docs/install/windows-line-endings.md

Windows and WSL2 line endings

ThothII's containers execute shell scripts from the source checkout. Those files must stay LF, even when the PC normally uses CRLF. The repository's .gitattributes is authoritative, but a Windows Git setting or an old checkout can still leave incorrect bytes. Check line endings after every clone and pull, before building an image.

Recommended WSL2 clone

Use Docker Desktop with WSL2 integration. Clone inside the Linux filesystem, for example under /home/<user>/src, rather than under /mnt/c. This avoids slow cross-filesystem builds, permission surprises, and Windows tools rewriting files behind WSL.

mkdir -p "$HOME/src"
cd "$HOME/src"
git -c core.autocrlf=false clone https://github.example.invalid/your-org/ThothII.git
cd ThothII
git config --local core.autocrlf false
bash scripts/verify-line-endings.sh

Keep Docker Desktop's integration enabled for that WSL distribution. Run the Linux build scripts and the Linux thothctl binary from the same WSL shell.

Repository-local LF policy

Set the option in this repository only. Do not change a company-wide or personal Git policy just for ThothII.

git config --local core.autocrlf false
git config --local --get core.autocrlf

The second command must print false. .gitattributes keeps shell, YAML, Dockerfile, JSON, TypeScript, Python, and Markdown files at LF; PowerShell files remain CRLF.

For a native PowerShell clone, disable conversion during the first checkout and then store the repository-local setting:

git -c core.autocrlf=false clone https://github.example.invalid/your-org/ThothII.git
Set-Location ThothII
git config --local core.autocrlf false
& "C:\Program Files\Git\bin\bash.exe" scripts/verify-line-endings.sh

Verify after clone or pull

From WSL2, Git Bash, macOS, or Linux run:

bash scripts/verify-line-endings.sh

Success exits with code 0 and prints no offending path. If it lists a file, do not build or start ThothII. Correct the checkout first. Native PowerShell users can invoke the same script through Git for Windows as shown above.

Recover an existing CRLF clone

The safest recovery is to reclone into a new directory. First commit wanted work or copy it to a backup outside both clones. Then clone with conversion disabled, run the verifier, and copy back only reviewed changes.

If a reviewed working tree must be repaired in place, Git must first normalize the index, export that exact index to a separate repair directory, verify the exported bytes, and only then copy the verified tracked files over the worktree. git add --renormalize . alone does not change existing worktree bytes.

WARNING — destructive worktree rewrite. Make a backup outside the clone or commit every wanted tracked change before continuing. The copy step below overwrites tracked worktree bytes from the staged index export. Stop if the staged diff does not contain exactly the wanted content; untracked files are neither exported nor repaired.

From WSL2, Git Bash, macOS, or Linux:

set -euo pipefail

abort_repair() { printf 'CRLF repair stopped: %s\n' "$1" >&2; exit 1; }
validate_index_export() {
  git ls-files -s -z | while IFS= read -r -d '' entry; do
    metadata="${entry%%$'\t'*}"
    path="${entry#*$'\t'}"
    mode="${metadata%% *}"
    [[ "$path" != "$entry" ]] || exit 1
    case "$mode" in
      100644|100755) [[ -f "$REPAIR_DIR/$path" && ! -L "$REPAIR_DIR/$path" ]] || exit 1 ;;
      120000) [[ -L "$REPAIR_DIR/$path" ]] && readlink "$REPAIR_DIR/$path" >/dev/null || exit 1 ;;
      *) printf 'Unsupported Git mode %s: %s\n' "$mode" "$path" >&2; exit 1 ;;
    esac
  done
}
validate_worktree_modes() {
  git ls-files -s -z | while IFS= read -r -d '' entry; do
    metadata="${entry%%$'\t'*}"
    path="${entry#*$'\t'}"
    mode="${metadata%% *}"
    case "$mode" in
      100644|100755) [[ -f "$path" && ! -L "$path" ]] || exit 1 ;;
      120000) [[ -L "$path" ]] && readlink "$path" >/dev/null || exit 1 ;;
      *) exit 1 ;;
    esac
  done
}
rewrite_index_entry() {
  local mode="$1" path="$2" target temporary_link
  case "$mode" in
    100644)
      cp "$REPAIR_DIR/$path" "$path" && chmod a-x "$path"
      ;;
    100755)
      cp "$REPAIR_DIR/$path" "$path" && chmod a+x "$path"
      ;;
    120000)
      target="$(readlink "$REPAIR_DIR/$path")" || return 1
      temporary_link="${path}.thoth-lf-repair-link"
      [[ ! -e "$temporary_link" && ! -L "$temporary_link" ]] || return 1
      ln -s "$target" "$temporary_link" || return 1
      rm -f "$path" || { rm -f "$temporary_link"; return 1; }
      mv "$temporary_link" "$path"
      ;;
    *) return 1 ;;
  esac
}

if ! git status --short; then abort_repair "git status failed"; fi
if ! git config --local core.autocrlf false; then abort_repair "could not set repository LF policy"; fi
if ! git add --renormalize .; then abort_repair "index renormalization failed"; fi
if ! git diff --cached --check; then abort_repair "normalized index check failed"; fi
if ! git diff --cached; then abort_repair "normalized index review failed"; fi
REPAIR_DIR="$(cd .. && pwd -P)/ThothII-lf-repair"
if [[ -e "$REPAIR_DIR" ]]; then
  abort_repair "choose a new empty LF repair directory: $REPAIR_DIR"
fi
if ! mkdir -p "$REPAIR_DIR"; then abort_repair "could not create LF repair directory"; fi
REPAIR_PREFIX="$REPAIR_DIR/"
if ! git checkout-index --all --force --prefix="$REPAIR_PREFIX"; then abort_repair "index export failed"; fi
if ! validate_index_export; then abort_repair "index export is missing entries or Git modes"; fi
if ! bash scripts/verify-line-endings.sh "$REPAIR_DIR"; then abort_repair "exported bytes failed LF verification"; fi
# WARNING: destructive copy; make a backup or commit wanted changes before this command.
if ! git ls-files -s -z | while IFS= read -r -d '' entry; do
  metadata="${entry%%$'\t'*}"
  path="${entry#*$'\t'}"
  mode="${metadata%% *}"
  rewrite_index_entry "$mode" "$path" || exit 1
done; then
  abort_repair "tracked-file rewrite failed; do not build from this worktree"
fi
if ! validate_worktree_modes; then abort_repair "repaired worktree does not match Git index modes"; fi
if ! bash scripts/verify-line-endings.sh; then abort_repair "repaired worktree failed LF verification"; fi
if ! git diff --cached --check; then abort_repair "repaired index check failed"; fi

Native Windows PowerShell runs the same Git operations and invokes the byte verifier through Git for Windows:

$ErrorActionPreference = 'Stop'
function Assert-NativeSuccess([string]$Step) {
  if ($LASTEXITCODE -ne 0) { throw "$Step failed with exit code $LASTEXITCODE." }
}
function ConvertFrom-IndexEntry([string]$Entry) {
  if ($Entry -notmatch '^([0-9]{6}) [0-9a-f]+ [0-3]\t(.+)$') {
    throw "Invalid Git index entry: $Entry"
  }
  [pscustomobject]@{ Mode = $Matches[1]; Path = $Matches[2] }
}

git status --short
Assert-NativeSuccess 'git status'
git config --local core.autocrlf false
Assert-NativeSuccess 'repository LF policy'
git add --renormalize .
Assert-NativeSuccess 'index renormalization'
git diff --cached --check
Assert-NativeSuccess 'normalized index check'
git diff --cached
Assert-NativeSuccess 'normalized index review'
$RepairDir = Join-Path (Split-Path -Parent (Get-Location).Path) 'ThothII-lf-repair'
if (Test-Path $RepairDir) { throw 'Choose a new empty LF repair directory.' }
New-Item -ItemType Directory -Path $RepairDir | Out-Null
$RepairPrefix = $RepairDir.Replace('\', '/') + '/'
git -c core.symlinks=true checkout-index --all --force --prefix=$RepairPrefix
Assert-NativeSuccess 'index export'
$RawIndexEntries = @(git ls-files -s)
Assert-NativeSuccess 'index inventory'
$IndexEntries = @($RawIndexEntries | ForEach-Object { ConvertFrom-IndexEntry $_ })
foreach ($Entry in $IndexEntries) {
  $ExportPath = Join-Path $RepairDir $Entry.Path
  $ExportItem = Get-Item -LiteralPath $ExportPath -Force -ErrorAction Stop
  switch ($Entry.Mode) {
    { $_ -in '100644', '100755' } {
      if ($ExportItem.LinkType -eq 'SymbolicLink') { throw "Regular export became a symlink: $($Entry.Path)" }
    }
    '120000' {
      if ($ExportItem.LinkType -ne 'SymbolicLink') { throw "Symlink export is not mode 120000: $($Entry.Path)" }
      if ([string]::IsNullOrWhiteSpace([string]$ExportItem.Target)) { throw "Symlink target is empty: $($Entry.Path)" }
    }
    default { throw "Unsupported Git mode $($Entry.Mode): $($Entry.Path)" }
  }
}
& "C:\Program Files\Git\bin\bash.exe" scripts/verify-line-endings.sh $RepairDir
Assert-NativeSuccess 'exported byte LF verification'
# WARNING: destructive copy; make a backup or commit wanted changes before this command.
foreach ($Entry in $IndexEntries) {
  $ExportPath = Join-Path $RepairDir $Entry.Path
  switch ($Entry.Mode) {
    { $_ -in '100644', '100755' } {
      Copy-Item -LiteralPath $ExportPath -Destination $Entry.Path -Force -ErrorAction Stop
    }
    '120000' {
      $LinkTarget = [string](Get-Item -LiteralPath $ExportPath -Force -ErrorAction Stop).Target
      $TemporaryLink = "$($Entry.Path).thoth-lf-repair-link"
      if (Test-Path -LiteralPath $TemporaryLink) { throw "Temporary symlink path exists: $TemporaryLink" }
      New-Item -ItemType SymbolicLink -Path $TemporaryLink -Target $LinkTarget -ErrorAction Stop | Out-Null
      Remove-Item -LiteralPath $Entry.Path -Force -ErrorAction Stop
      Move-Item -LiteralPath $TemporaryLink -Destination $Entry.Path -ErrorAction Stop
    }
    default { throw "Unsupported Git mode $($Entry.Mode): $($Entry.Path)" }
  }
}
foreach ($Entry in $IndexEntries) {
  $WorktreeItem = Get-Item -LiteralPath $Entry.Path -Force -ErrorAction Stop
  switch ($Entry.Mode) {
    { $_ -in '100644', '100755' } {
      if ($WorktreeItem.LinkType -eq 'SymbolicLink') { throw "Regular worktree entry became a symlink: $($Entry.Path)" }
    }
    '120000' {
      if ($WorktreeItem.LinkType -ne 'SymbolicLink') { throw "Repaired worktree symlink is not mode 120000: $($Entry.Path)" }
      if ([string]::IsNullOrWhiteSpace([string]$WorktreeItem.Target)) { throw "Repaired symlink target is empty: $($Entry.Path)" }
    }
    default { throw "Unsupported Git mode $($Entry.Mode): $($Entry.Path)" }
  }
}
& "C:\Program Files\Git\bin\bash.exe" scripts/verify-line-endings.sh
Assert-NativeSuccess 'repaired worktree LF verification'
git diff --cached --check
Assert-NativeSuccess 'repaired index check'

The export inventory must contain every regular mode (100644/100755) and recreate every tracked workspace compatibility symlink (120000). The first verifier proves the complete export before any overwrite; every copy/link operation is fail-closed; the final verifier examines the repaired worktree bytes. On native Windows, creating symlinks requires Developer Mode or an elevated account; failure stops the rewrite. Review the staged diff again before committing, then remove the separate repair directory only after inspecting it. The procedure intentionally avoids git reset --hard; replacing the clone is easier to audit and safer for uncommitted work.