Zenodo, 2026 · Muhammad Maroof

QuerySmith: Fine-Tuning Phi-4-mini for Text-to-SQL Generation with CPU-Efficient Partial-Residual Quantization

LLMs fine-tuned for text-to-SQL are usually evaluated and deployed assuming GPU inference, which limits use in on-premise or resource-constrained settings. This work (1) fine-tunes Phi-4-mini-instruct (3.8B parameters) with QLoRA (rank 16, alpha 32, dropout 0.05) on the QuerySmith-Spider-Bird dataset, and (2) introduces a progression of CPU quantization strategies, culminating in a partial-residual quantization scheme that keeps accuracy close to FP32 while running on CPU-only hardware.

Key contributions

• Compact text-to-SQL model fine-tuned on a Spider/BIRD-derived dataset
• Partial-residual quantization: aggressive quantization of a subset of transformer blocks plus low-rank SVD error correction; recovers 92.2% of the accuracy lost by naive INT4 (0.707 vs 0.636)
• llama.cpp / GGUF deployment adds a further 65.3% latency reduction; best throughput at 8 to 12 CPU threads on a 14-core Intel Core i5 (16 GB RAM)
• Post-optimization fine-tuning exceeds the FP32 baseline (0.743 vs 0.713) while running 6.6x faster and 4.2x smaller

Limitations: Residual rank fixed at 8; 200-example evaluation set.

@misc{maroof2026querysmith,
  title={QuerySmith: Fine-Tuning Phi-4-mini for Text-to-SQL Generation with CPU-Efficient Partial-Residual Quantization},
  author={Maroof, Muhammad},
  year={2026},
  publisher={Zenodo},
  url={https://zenodo.org/records/23000490}
}

Zenodo preprint, May 2026 · Muhammad Maroof

An Empirical Study of Adaptive Activation Attention

This paper introduces Adaptive Activation Attention (A³), a method that replaces the fixed softmax in Transformer attention with a learnable, input-dependent activation. We prove that A³ always produces a valid probability distribution and that, with default initialization, it exactly recovers standard softmax attention at the start of training.

Key contributions

• Proposes A³: a learnable attention activation that adapts per input using a gated score matrix and a learned additive bias
• Proves validity: outputs are non-negative and sum to 1 for any parameter values
• Proves softmax recovery: default initialization behaves exactly like standard softmax attention
• Empirically studies the behavior and potential of adaptive activation attention in Transformer models

@misc{maroof2026empirical,
  title={An Empirical Study of Adaptive Activation Attention},
  author={Maroof, Muhammad},
  year={2026},
  month={may},
  howpublished={Zenodo preprint, version 1.0},
  publisher={Zenodo},
  url={https://zenodo.org/records/20364414}
}