diff --git a/benchmarks/README.md b/benchmarks/README.md index bde6fb26..ce048005 100644 --- a/benchmarks/README.md +++ b/benchmarks/README.md @@ -2,31 +2,73 @@ This directory contains benchmark scripts for testing the performance of various database operations using `pyodbc` and `mssql_python`. The goal is to evaluate and compare the performance of these libraries for common database operations. +## Benchmark Scripts + +### 1. `bench_mssql.py` - Richbench Framework Benchmarks +Comprehensive benchmarks using the richbench framework for detailed performance analysis. + +### 2. `perf-benchmarking.py` - Real-World Query Benchmarks +Standalone script that tests real-world queries against AdventureWorks2022 database with statistical analysis. + ## Why Benchmarks? - To measure the efficiency of `pyodbc` and `mssql_python` in handling database operations. - To identify performance bottlenecks and optimize database interactions. - To ensure the reliability and scalability of the libraries under different workloads. ## How to Run Benchmarks + +### Running bench_mssql.py (Richbench Framework) + 1. **Set Up the Environment Variable**: - Ensure you have a running SQL Server instance. - Set the `DB_CONNECTION_STRING` environment variable with the connection string to your database. For example: - ```cmd - set DB_CONNECTION_STRING=Server=your_server;Database=your_database;UID=your_user;PWD=your_password; + ```bash + export DB_CONNECTION_STRING="Server=your_server;Database=AdventureWorks2022;UID=your_user;PWD=your_password;" ``` 2. **Install Richbench - Benchmarking Tool**: - - Install richbench : - ```cmd - pip install richbench - ``` + ```bash + pip install richbench + ``` 3. **Run the Benchmarks**: - - Execute richbench from the parent folder (mssql-python) : - ```cmd + - Execute richbench from the parent folder (mssql-python): + ```bash richbench benchmarks ``` - Results will be displayed in the terminal with detailed performance metrics. + - Results will be displayed in the terminal with detailed performance metrics. + +### Running perf-benchmarking.py (Real-World Queries) + +This script tests performance with real-world queries from the AdventureWorks2022 database. + +1. **Prerequisites**: + - AdventureWorks2022 database must be available + - Both `pyodbc` and `mssql-python` must be installed + - Update the connection string in the script if needed + +2. **Run from project root**: + ```bash + python benchmarks/perf-benchmarking.py + ``` + +3. **Features**: + - Runs each query multiple times (default: 5 iterations) + - Calculates average, min, max, and standard deviation + - Provides speedup comparisons between libraries + - Tests various query patterns: + - Complex joins with aggregations + - Large dataset retrieval (10K+ rows) + - Very large dataset (1.2M rows) + - CTEs and subqueries + - Detailed summary tables and conclusions + +4. **Output**: + The script provides: + - Progress indicators during execution + - Detailed results for each benchmark + - Summary comparison table + - Overall performance conclusion with speedup factors ## Key Features of `bench_mssql.py` - **Comprehensive Benchmarks**: Includes SELECT, INSERT, UPDATE, DELETE, complex queries, stored procedures, and transaction handling. @@ -34,7 +76,15 @@ This directory contains benchmark scripts for testing the performance of various - **Progress Messages**: Clear progress messages are printed during execution for better visibility. - **Automated Setup and Cleanup**: The script automatically sets up and cleans up the database environment before and after the benchmarks. +## Key Features of `perf-benchmarking.py` +- **Statistical Analysis**: Multiple iterations with avg/min/max/stddev calculations +- **Real-World Queries**: Tests against AdventureWorks2022 with production-like queries +- **Automatic Import Resolution**: Correctly imports local `mssql_python` package +- **Comprehensive Reporting**: Detailed comparison tables and performance summaries +- **Speedup Calculations**: Clear indication of performance differences + ## Notes - Ensure the database user has the necessary permissions to create and drop tables and stored procedures. -- The script uses permanent tables prefixed with `perfbenchmark_` for benchmarking purposes. -- A stored procedure named `perfbenchmark_stored_procedure` is created and used during the benchmarks. \ No newline at end of file +- The `bench_mssql.py` script uses permanent tables prefixed with `perfbenchmark_` for benchmarking purposes. +- A stored procedure named `perfbenchmark_stored_procedure` is created and used during the benchmarks. +- The `perf-benchmarking.py` script connects to AdventureWorks2022 and requires read permissions only. \ No newline at end of file diff --git a/benchmarks/perf-benchmarking.py b/benchmarks/perf-benchmarking.py new file mode 100644 index 00000000..b606b1d8 --- /dev/null +++ b/benchmarks/perf-benchmarking.py @@ -0,0 +1,359 @@ +""" +Performance Benchmarking Script for mssql-python vs pyodbc + +This script runs comprehensive performance tests comparing mssql-python with pyodbc +across multiple query types and scenarios. Each test is run multiple times to calculate +average execution times, minimum, maximum, and standard deviation. + +Usage: + python benchmarks/perf-benchmarking.py + +Requirements: + - pyodbc + - mssql_python + - Valid SQL Server connection +""" + +import os +import sys +import time +import statistics +from typing import List, Tuple + +# Add parent directory to path to import local mssql_python +sys.path.insert(0, os.path.abspath(os.path.join(os.path.dirname(__file__), '..'))) + +import pyodbc +from mssql_python import connect + +# Configuration +CONN_STR = os.getenv("DB_CONNECTION_STRING") + +if not CONN_STR: + print("Error: The environment variable DB_CONNECTION_STRING is not set. Please set it to a valid SQL Server connection string and try again.") + sys.exit(1) +# Ensure pyodbc connection string has ODBC driver specified +if CONN_STR and 'Driver=' not in CONN_STR: + CONN_STR = f"Driver={{ODBC Driver 18 for SQL Server}};{CONN_STR}" + +NUM_ITERATIONS = 5 # Number of times to run each test for averaging + +# SQL Queries +COMPLEX_JOIN_AGGREGATION = """ + SELECT + p.ProductID, + p.Name AS ProductName, + pc.Name AS Category, + psc.Name AS Subcategory, + COUNT(sod.SalesOrderDetailID) AS TotalOrders, + SUM(sod.OrderQty) AS TotalQuantity, + SUM(sod.LineTotal) AS TotalRevenue, + AVG(sod.UnitPrice) AS AvgPrice + FROM Sales.SalesOrderDetail sod + INNER JOIN Production.Product p ON sod.ProductID = p.ProductID + INNER JOIN Production.ProductSubcategory psc ON p.ProductSubcategoryID = psc.ProductSubcategoryID + INNER JOIN Production.ProductCategory pc ON psc.ProductCategoryID = pc.ProductCategoryID + GROUP BY p.ProductID, p.Name, pc.Name, psc.Name + HAVING SUM(sod.LineTotal) > 10000 + ORDER BY TotalRevenue DESC; +""" + +LARGE_DATASET = """ + SELECT + soh.SalesOrderID, + soh.OrderDate, + soh.DueDate, + soh.ShipDate, + soh.Status, + soh.SubTotal, + soh.TaxAmt, + soh.Freight, + soh.TotalDue, + c.CustomerID, + p.FirstName, + p.LastName, + a.AddressLine1, + a.City, + sp.Name AS StateProvince, + cr.Name AS Country + FROM Sales.SalesOrderHeader soh + INNER JOIN Sales.Customer c ON soh.CustomerID = c.CustomerID + INNER JOIN Person.Person p ON c.PersonID = p.BusinessEntityID + INNER JOIN Person.BusinessEntityAddress bea ON p.BusinessEntityID = bea.BusinessEntityID + INNER JOIN Person.Address a ON bea.AddressID = a.AddressID + INNER JOIN Person.StateProvince sp ON a.StateProvinceID = sp.StateProvinceID + INNER JOIN Person.CountryRegion cr ON sp.CountryRegionCode = cr.CountryRegionCode + WHERE soh.OrderDate >= '2013-01-01'; +""" + +VERY_LARGE_DATASET = """ +SELECT + sod.SalesOrderID, + sod.SalesOrderDetailID, + sod.ProductID, + sod.OrderQty, + sod.UnitPrice, + sod.LineTotal, + p.Name AS ProductName, + p.ProductNumber, + p.Color, + p.ListPrice, + n1.number AS RowMultiplier1 +FROM Sales.SalesOrderDetail sod +CROSS JOIN (SELECT TOP 10 ROW_NUMBER() OVER (ORDER BY (SELECT NULL)) AS number + FROM Sales.SalesOrderDetail) n1 +INNER JOIN Production.Product p ON sod.ProductID = p.ProductID; +""" + +SUBQUERY_WITH_CTE = """ + WITH SalesSummary AS ( + SELECT + soh.SalesPersonID, + YEAR(soh.OrderDate) AS OrderYear, + SUM(soh.TotalDue) AS YearlyTotal + FROM Sales.SalesOrderHeader soh + WHERE soh.SalesPersonID IS NOT NULL + GROUP BY soh.SalesPersonID, YEAR(soh.OrderDate) + ), + RankedSales AS ( + SELECT + SalesPersonID, + OrderYear, + YearlyTotal, + RANK() OVER (PARTITION BY OrderYear ORDER BY YearlyTotal DESC) AS SalesRank + FROM SalesSummary + ) + SELECT + rs.SalesPersonID, + p.FirstName, + p.LastName, + rs.OrderYear, + rs.YearlyTotal, + rs.SalesRank + FROM RankedSales rs + INNER JOIN Person.Person p ON rs.SalesPersonID = p.BusinessEntityID + WHERE rs.SalesRank <= 10 + ORDER BY rs.OrderYear DESC, rs.SalesRank; +""" + + +class BenchmarkResult: + """Class to store and calculate benchmark statistics""" + + def __init__(self, name: str): + self.name = name + self.times: List[float] = [] + self.row_count: int = 0 + + def add_time(self, elapsed: float, rows: int = 0): + """Add a timing result""" + self.times.append(elapsed) + if rows > 0: + self.row_count = rows + + @property + def avg_time(self) -> float: + """Calculate average time""" + return statistics.mean(self.times) if self.times else 0.0 + + @property + def min_time(self) -> float: + """Get minimum time""" + return min(self.times) if self.times else 0.0 + + @property + def max_time(self) -> float: + """Get maximum time""" + return max(self.times) if self.times else 0.0 + + @property + def std_dev(self) -> float: + """Calculate standard deviation""" + return statistics.stdev(self.times) if len(self.times) > 1 else 0.0 + + def __str__(self) -> str: + """Format results as string""" + return (f"{self.name}:\n" + f" Avg: {self.avg_time:.4f}s | Min: {self.min_time:.4f}s | " + f"Max: {self.max_time:.4f}s | StdDev: {self.std_dev:.4f}s | " + f"Rows: {self.row_count}") + + +def run_benchmark_pyodbc(query: str, name: str, iterations: int) -> BenchmarkResult: + """Run a benchmark using pyodbc""" + result = BenchmarkResult(f"{name} (pyodbc)") + + for i in range(iterations): + try: + start_time = time.time() + conn = pyodbc.connect(CONN_STR) + cursor = conn.cursor() + cursor.execute(query) + rows = cursor.fetchall() + elapsed = time.time() - start_time + + result.add_time(elapsed, len(rows)) + + cursor.close() + conn.close() + except Exception as e: + print(f" Error in iteration {i+1}: {e}") + continue + + return result + + +def run_benchmark_mssql_python(query: str, name: str, iterations: int) -> BenchmarkResult: + """Run a benchmark using mssql-python""" + result = BenchmarkResult(f"{name} (mssql-python)") + + for i in range(iterations): + try: + start_time = time.time() + conn = connect(CONN_STR) + cursor = conn.cursor() + cursor.execute(query) + rows = cursor.fetchall() + elapsed = time.time() - start_time + + result.add_time(elapsed, len(rows)) + + cursor.close() + conn.close() + except Exception as e: + print(f" Error in iteration {i+1}: {e}") + continue + + return result + + +def calculate_speedup(pyodbc_result: BenchmarkResult, mssql_python_result: BenchmarkResult) -> float: + """Calculate speedup factor""" + if mssql_python_result.avg_time == 0: + return 0.0 + return pyodbc_result.avg_time / mssql_python_result.avg_time + + +def print_comparison(pyodbc_result: BenchmarkResult, mssql_python_result: BenchmarkResult): + """Print detailed comparison of results""" + speedup = calculate_speedup(pyodbc_result, mssql_python_result) + + print(f"\n{'='*80}") + print(f"BENCHMARK: {pyodbc_result.name.split(' (')[0]}") + print(f"{'='*80}") + print(f"\npyodbc:") + print(f" Avg: {pyodbc_result.avg_time:.4f}s") + print(f" Min: {pyodbc_result.min_time:.4f}s") + print(f" Max: {pyodbc_result.max_time:.4f}s") + print(f" StdDev: {pyodbc_result.std_dev:.4f}s") + print(f" Rows: {pyodbc_result.row_count}") + + print(f"\nmssql-python:") + print(f" Avg: {mssql_python_result.avg_time:.4f}s") + print(f" Min: {mssql_python_result.min_time:.4f}s") + print(f" Max: {mssql_python_result.max_time:.4f}s") + print(f" StdDev: {mssql_python_result.std_dev:.4f}s") + print(f" Rows: {mssql_python_result.row_count}") + + print(f"\nPerformance:") + if speedup > 1: + print(f" mssql-python is {speedup:.2f}x FASTER than pyodbc") + elif speedup < 1 and speedup > 0: + print(f" mssql-python is {1/speedup:.2f}x SLOWER than pyodbc") + else: + print(f" Unable to calculate speedup") + + print(f" Time difference: {(pyodbc_result.avg_time - mssql_python_result.avg_time):.4f}s") + + +def main(): + """Main benchmark runner""" + print("="*80) + print("PERFORMANCE BENCHMARKING: mssql-python vs pyodbc") + print("="*80) + print(f"\nConfiguration:") + print(f" Iterations per test: {NUM_ITERATIONS}") + print(f" Database: AdventureWorks2022") + print(f"\n") + + # Define benchmarks + benchmarks = [ + (COMPLEX_JOIN_AGGREGATION, "Complex Join Aggregation"), + (LARGE_DATASET, "Large Dataset Retrieval"), + (VERY_LARGE_DATASET, "Very Large Dataset (1.2M rows)"), + (SUBQUERY_WITH_CTE, "Subquery with CTE"), + ] + + # Store all results for summary + all_results: List[Tuple[BenchmarkResult, BenchmarkResult]] = [] + + # Run each benchmark + for query, name in benchmarks: + print(f"\nRunning: {name}") + print(f" Testing with pyodbc... ", end="", flush=True) + pyodbc_result = run_benchmark_pyodbc(query, name, NUM_ITERATIONS) + print(f"OK (avg: {pyodbc_result.avg_time:.4f}s)") + + print(f" Testing with mssql-python... ", end="", flush=True) + mssql_python_result = run_benchmark_mssql_python(query, name, NUM_ITERATIONS) + print(f"OK (avg: {mssql_python_result.avg_time:.4f}s)") + + all_results.append((pyodbc_result, mssql_python_result)) + + # Print detailed comparisons + print("\n\n" + "="*80) + print("DETAILED RESULTS") + print("="*80) + + for pyodbc_result, mssql_python_result in all_results: + print_comparison(pyodbc_result, mssql_python_result) + + # Print summary table + print("\n\n" + "="*80) + print("SUMMARY TABLE") + print("="*80) + print(f"\n{'Benchmark':<35} {'pyodbc (s)':<15} {'mssql-python (s)':<20} {'Speedup'}") + print("-" * 80) + + total_pyodbc = 0.0 + total_mssql_python = 0.0 + + for pyodbc_result, mssql_python_result in all_results: + name = pyodbc_result.name.split(' (')[0] + speedup = calculate_speedup(pyodbc_result, mssql_python_result) + + total_pyodbc += pyodbc_result.avg_time + total_mssql_python += mssql_python_result.avg_time + + print(f"{name:<35} {pyodbc_result.avg_time:<15.4f} {mssql_python_result.avg_time:<20.4f} {speedup:.2f}x") + + print("-" * 80) + print(f"{'TOTAL':<35} {total_pyodbc:<15.4f} {total_mssql_python:<20.4f} " + f"{total_pyodbc/total_mssql_python if total_mssql_python > 0 else 0:.2f}x") + + # Overall conclusion + overall_speedup = total_pyodbc / total_mssql_python if total_mssql_python > 0 else 0 + print(f"\n{'='*80}") + print("OVERALL CONCLUSION") + print("="*80) + if overall_speedup > 1: + print(f"\nmssql-python is {overall_speedup:.2f}x FASTER than pyodbc on average") + print(f"Total time saved: {total_pyodbc - total_mssql_python:.4f}s ({((total_pyodbc - total_mssql_python)/total_pyodbc*100):.1f}%)") + elif overall_speedup < 1 and overall_speedup > 0: + print(f"\nmssql-python is {1/overall_speedup:.2f}x SLOWER than pyodbc on average") + print(f"Total time difference: {total_mssql_python - total_pyodbc:.4f}s ({((total_mssql_python - total_pyodbc)/total_mssql_python*100):.1f}%)") + + print(f"\n{'='*80}\n") + + +if __name__ == "__main__": + try: + main() + except KeyboardInterrupt: + print("\n\nBenchmark interrupted by user.") + sys.exit(1) + except Exception as e: + print(f"\n\nFatal error: {e}") + import traceback + traceback.print_exc() + sys.exit(1) diff --git a/eng/pipelines/pr-validation-pipeline.yml b/eng/pipelines/pr-validation-pipeline.yml index 401b4cf1..14e07885 100644 --- a/eng/pipelines/pr-validation-pipeline.yml +++ b/eng/pipelines/pr-validation-pipeline.yml @@ -180,6 +180,138 @@ jobs: env: DB_CONNECTION_STRING: 'Server=localhost;Database=TestDB;Uid=testuser;Pwd=$(DB_PASSWORD);TrustServerCertificate=yes' + # Download and restore AdventureWorks2022 database for benchmarking + - powershell: | + Write-Host "Downloading AdventureWorks2022.bak..." + $ProgressPreference = 'SilentlyContinue' + Invoke-WebRequest -Uri "https://github.com/Microsoft/sql-server-samples/releases/download/adventureworks/AdventureWorks2022.bak" -OutFile "$env:TEMP\AdventureWorks2022.bak" + + Write-Host "Restoring AdventureWorks2022 database..." + # Get the default data and log paths + $dataPath = sqlcmd -S "localhost" -U "sa" -P "$(DB_PASSWORD)" -Q "SET NOCOUNT ON; SELECT SERVERPROPERTY('InstanceDefaultDataPath') AS DataPath" -h -1 -C | Out-String + $logPath = sqlcmd -S "localhost" -U "sa" -P "$(DB_PASSWORD)" -Q "SET NOCOUNT ON; SELECT SERVERPROPERTY('InstanceDefaultLogPath') AS LogPath" -h -1 -C | Out-String + + $dataPath = $dataPath.Trim() + $logPath = $logPath.Trim() + + Write-Host "Data path: $dataPath" + Write-Host "Log path: $logPath" + + # Restore the database + sqlcmd -S "localhost" -U "sa" -P "$(DB_PASSWORD)" -C -Q @" + RESTORE DATABASE AdventureWorks2022 + FROM DISK = '$env:TEMP\AdventureWorks2022.bak' + WITH + MOVE 'AdventureWorks2022' TO '${dataPath}AdventureWorks2022.mdf', + MOVE 'AdventureWorks2022_log' TO '${logPath}AdventureWorks2022_log.ldf', + REPLACE + "@ + + if ($LASTEXITCODE -eq 0) { + Write-Host "AdventureWorks2022 database restored successfully" + } else { + Write-Error "Failed to restore AdventureWorks2022 database" + exit 1 + } + displayName: 'Download and restore AdventureWorks2022 database' + condition: eq(variables['sqlVersion'], 'SQL2022') + env: + DB_PASSWORD: $(DB_PASSWORD) + + # Run performance benchmarks on SQL Server 2022 + - powershell: | + Write-Host "Checking and installing ODBC Driver 18 for SQL Server..." + + # Check if ODBC Driver 18 is registered in Windows registry + $odbcDriverKey = "HKLM:\SOFTWARE\ODBC\ODBCINST.INI\ODBC Driver 18 for SQL Server" + $driverExists = Test-Path $odbcDriverKey + + if ($driverExists) { + Write-Host "✓ ODBC Driver 18 for SQL Server is already installed and registered" + $driverPath = (Get-ItemProperty -Path $odbcDriverKey -Name "Driver" -ErrorAction SilentlyContinue).Driver + if ($driverPath) { + Write-Host " Driver location: $driverPath" + } + } else { + Write-Host "ODBC Driver 18 for SQL Server not found, installing..." + + # Download ODBC Driver 18.5.2.1 (x64) from official Microsoft link + $ProgressPreference = 'SilentlyContinue' + $installerUrl = "https://go.microsoft.com/fwlink/?linkid=2335671" + $installerPath = "$env:TEMP\msodbcsql_18.5.2.1_x64.msi" + + Write-Host "Downloading ODBC Driver 18 (x64) from Microsoft..." + Write-Host " URL: $installerUrl" + try { + Invoke-WebRequest -Uri $installerUrl -OutFile $installerPath -UseBasicParsing + Write-Host "✓ Download completed: $installerPath" + } catch { + Write-Error "Failed to download ODBC driver: $_" + exit 1 + } + + Write-Host "Installing ODBC Driver 18..." + $installArgs = @( + "/i" + "`"$installerPath`"" + "/quiet" + "/qn" + "/norestart" + "IACCEPTMSODBCSQLLICENSETERMS=YES" + "/l*v" + "`"$env:TEMP\odbc_install.log`"" + ) + + $installCmd = "msiexec.exe $($installArgs -join ' ')" + Write-Host " Command: $installCmd" + + $process = Start-Process msiexec.exe -ArgumentList $installArgs -Wait -PassThru -NoNewWindow + + if ($process.ExitCode -eq 0) { + Write-Host "✓ ODBC Driver 18 installation completed successfully" + } elseif ($process.ExitCode -eq 3010) { + Write-Host "✓ ODBC Driver 18 installed (reboot recommended but not required)" + } else { + Write-Error "ODBC Driver 18 installation failed with exit code: $($process.ExitCode)" + Write-Host "Check installation log: $env:TEMP\odbc_install.log" + Get-Content "$env:TEMP\odbc_install.log" -Tail 50 -ErrorAction SilentlyContinue + exit 1 + } + + # Wait for registry update + Start-Sleep -Seconds 2 + + # Clean up installer + Remove-Item $installerPath -ErrorAction SilentlyContinue + } + + # Final verification using registry + Write-Host "`nVerifying ODBC Driver 18 installation..." + $verifyKey = Test-Path "HKLM:\SOFTWARE\ODBC\ODBCINST.INI\ODBC Driver 18 for SQL Server" + + if ($verifyKey) { + $driverInfo = Get-ItemProperty -Path "HKLM:\SOFTWARE\ODBC\ODBCINST.INI\ODBC Driver 18 for SQL Server" -ErrorAction SilentlyContinue + Write-Host "✓ SUCCESS: ODBC Driver 18 for SQL Server is registered" + Write-Host " Driver: $($driverInfo.Driver)" + Write-Host " Setup: $($driverInfo.Setup)" + } else { + Write-Error "ODBC Driver 18 for SQL Server is not registered in ODBC" + Write-Host "`nListing all installed ODBC drivers from registry:" + Get-ChildItem "HKLM:\SOFTWARE\ODBC\ODBCINST.INI" -ErrorAction SilentlyContinue | ForEach-Object { Write-Host " - $($_.PSChildName)" } + exit 1 + } + + Write-Host "`nInstalling pyodbc..." + pip install pyodbc + + Write-Host "`nRunning performance benchmarks..." + python benchmarks/perf-benchmarking.py + displayName: 'Run performance benchmarks on SQL Server 2022' + condition: eq(variables['sqlVersion'], 'SQL2022') + continueOnError: true + env: + DB_CONNECTION_STRING: 'Server=localhost;Database=AdventureWorks2022;Uid=sa;Pwd=$(DB_PASSWORD);TrustServerCertificate=yes' + - task: CopyFiles@2 inputs: SourceFolder: 'mssql_python' @@ -499,6 +631,92 @@ jobs: env: DB_PASSWORD: $(DB_PASSWORD) + - script: | + # Download and restore AdventureWorks2022 database for benchmarking on Ubuntu only + if [ "$(distroName)" = "Ubuntu" ] && [ "$(useAzureSQL)" = "false" ]; then + echo "Downloading AdventureWorks2022.bak..." + wget -q https://github.com/Microsoft/sql-server-samples/releases/download/adventureworks/AdventureWorks2022.bak -O /tmp/AdventureWorks2022.bak + + echo "Copying backup file into SQL Server container..." + docker cp /tmp/AdventureWorks2022.bak sqlserver-$(distroName):/tmp/AdventureWorks2022.bak + + echo "Restoring AdventureWorks2022 database..." + docker exec sqlserver-$(distroName) /opt/mssql-tools18/bin/sqlcmd \ + -S localhost \ + -U SA \ + -P "$(DB_PASSWORD)" \ + -C \ + -Q "RESTORE DATABASE AdventureWorks2022 FROM DISK = '/tmp/AdventureWorks2022.bak' WITH MOVE 'AdventureWorks2022' TO '/var/opt/mssql/data/AdventureWorks2022.mdf', MOVE 'AdventureWorks2022_log' TO '/var/opt/mssql/data/AdventureWorks2022_log.ldf', REPLACE" + + if [ $? -eq 0 ]; then + echo "AdventureWorks2022 database restored successfully" + else + echo "Failed to restore AdventureWorks2022 database" + fi + + # Clean up (ignore errors if files are locked) + rm -f /tmp/AdventureWorks2022.bak || true + docker exec sqlserver-$(distroName) rm -f /tmp/AdventureWorks2022.bak || true + fi + displayName: 'Download and restore AdventureWorks2022 database in $(distroName)' + condition: and(eq(variables['distroName'], 'Ubuntu'), eq(variables['useAzureSQL'], 'false')) + continueOnError: true + env: + DB_PASSWORD: $(DB_PASSWORD) + + - script: | + # Run performance benchmarks on Ubuntu with SQL Server 2022 only + if [ "$(distroName)" = "Ubuntu" ] && [ "$(useAzureSQL)" = "false" ]; then + SQLSERVER_IP=$(docker inspect sqlserver-$(distroName) --format='{{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}}') + echo "Running performance benchmarks on Ubuntu with SQL Server IP: $SQLSERVER_IP" + + docker exec \ + -e DB_CONNECTION_STRING="Server=$SQLSERVER_IP;Database=AdventureWorks2022;Uid=SA;Pwd=$(DB_PASSWORD);TrustServerCertificate=yes" \ + test-container-$(distroName) bash -c " + source /opt/venv/bin/activate + + echo 'Reinstalling ODBC Driver for benchmarking...' + export DEBIAN_FRONTEND=noninteractive + + # Remove duplicate repository sources if they exist + rm -f /etc/apt/sources.list.d/microsoft-prod.list + + # Add Microsoft repository + curl -sSL https://packages.microsoft.com/keys/microsoft.asc | apt-key add - + curl -sSL https://packages.microsoft.com/config/ubuntu/22.04/prod.list > /etc/apt/sources.list.d/mssql-release.list + + # Update package lists + apt-get update -qq + + # Install unixodbc and its dependencies first (provides libodbcinst.so.2 needed by msodbcsql18) + echo 'Installing unixODBC dependencies...' + apt-get install -y --no-install-recommends unixodbc unixodbc-dev libodbc1 odbcinst odbcinst1debian2 + + # Verify libodbcinst.so.2 is available + ldconfig + ls -la /usr/lib/x86_64-linux-gnu/libodbcinst.so.2 || echo 'Warning: libodbcinst.so.2 not found' + + # Install ODBC Driver 18 + echo 'Installing msodbcsql18...' + ACCEPT_EULA=Y apt-get install -y msodbcsql18 + + # Verify ODBC driver installation + odbcinst -q -d -n 'ODBC Driver 18 for SQL Server' || echo 'Warning: ODBC Driver 18 not registered' + + echo 'Installing pyodbc for benchmarking...' + pip install pyodbc + echo 'Running performance benchmarks on $(distroName)' + python benchmarks/perf-benchmarking.py || echo 'Performance benchmark failed or database not available' + " + else + echo "Skipping performance benchmarks on $(distroName) (only runs on Ubuntu with local SQL Server)" + fi + displayName: 'Run performance benchmarks in $(distroName) container' + condition: and(eq(variables['distroName'], 'Ubuntu'), eq(variables['useAzureSQL'], 'false')) + continueOnError: true + env: + DB_PASSWORD: $(DB_PASSWORD) + - script: | # Copy test results from container to host docker cp test-container-$(distroName):/workspace/test-results-$(distroName).xml $(Build.SourcesDirectory)/