Quantum-Enhanced Multi-Head Attention for Hybrid Classical-Quantum Neural Networks
Description
This code integrates quantum-inspired computations into a deep learning pipeline to classify images from the CIFAR-10 dataset. It combines traditional neural network layers with a quantum multi-head attention mechanism, leveraging the capabilities of both classical and quantum approaches. The model utilizes PyTorch for defining and training the network, while Qiskit and Qiskit Aer simulate quantum circuits for the quantum-inspired components. The CIFAR-10 dataset is preprocessed using torchvision, with subsets created for efficient training and validation. The quantum multi-head attention mechanism is implemented as a custom PyTorch autograd function, QuantumMultiHeadAttentionFunction. This function defines the forward and backward passes, where data is encoded into quantum circuits and parameter updates are computed using finite-difference approximation. The function relies on Z-expectation values of qubits, simulated using Qiskit Aer's statevector simulator. These computations are encapsulated in the QuantumMultiHeadAttentionLayer, which integrates seamlessly with PyTorch's module structure. The core model, HybridAttentionCIFAR, consists of a combination of convolutional layers, a linear embedding layer, the quantum attention layer, and a fully connected output layer. The convolutional layers extract features from input images, which are then transformed into embeddings compatible with the quantum attention layer. The quantum attention mechanism processes these embeddings, enhancing feature representation, and the final fully connected layer maps the processed embeddings to class logits for classification. The get_cifar_subset function prepares the CIFAR-10 data, creating small subsets for training and validation. These subsets are batched using PyTorch's DataLoader. The train_cifar_qattn function orchestrates the training process, configuring the model, optimizer, and loss function. Over multiple epochs, it computes training and validation losses and accuracies, updating model parameters using the Adam optimizer.