Day 11: Export File Generation & Blob Storage Integration

By Day 10, we had observability fully wired across our API and isolated Function App, catching silent integration bugs before they could touch our production environment. However, up to this point, our asynchronous pipeline remained conceptual—the system tracked jobs and processed queues, but no actual data was being delivered.

Today, we closed the loop. Our background worker now queries live, real-world task data from Cosmos DB, generates fully formatted, injection-safe CSV files, streams them directly to Azure Blob Storage, and hands off a secure, temporary download link back to the frontend.

Why Real-World File Generation Matters

In an enterprise platform, file export utilities are a core business requirement. By building an event-driven, secure background export pipeline, we prove several key architectural principles:

Asynchronous Offloading: Long-running, resource-intensive operations (like querying thousands of database rows and parsing them into files) are handled completely outside the user's HTTP request/response thread.
Storage Tier Isolation: Files are written directly to high-scale, cost-effective object storage (Blob Storage) rather than occupying disk space on our application servers.
Zero-Trust File Delivery: Files are never stored in a public container. Instead, we gate access using short-lived, identity-backed tokens.

Step 1: Extending the Function Worker with Dependency Injection

To allow our background worker to query Cosmos DB and write directly to Blob Storage, we first updated our isolated Function App's Program.cs to inject BlobServiceClient and ITaskRepository using the same passwordless DefaultAzureCredential configured earlier.

The Namespace & Access Gotcha: When bridging separate assemblies, ensure that your interface (like ITaskRepository) is declared as public rather than default internal. We also resolved a common namespace pitfall by importing backend.Interfaces inside our worker to properly map our database operations.

With the dependency injection chain established, we refactored ProcessExportJob.cs to map to our actual production TaskItem model, build an in-memory CSV string, and stream it straight to our secure container:

using System;
using System.Text;
using System.Text.Json;
using System.Threading.Tasks;
using System.Linq;
using System.IO;
using System.Collections.Generic;
using Azure.Storage.Queues.Models;
using Azure.Storage.Blobs;
using Microsoft.Azure.Functions.Worker;
using Microsoft.Extensions.Logging;
using backend.Interfaces; // Imported to resolve our repository contracts
using backend.Models;     // Imported to resolve our TaskItem model

namespace Company.Function;

public class ProcessExportJob
{
    private readonly ILogger<ProcessExportJob> _logger;
    private readonly IExportRepository _exportRepository;
    private readonly ITaskRepository _taskRepository; 
    private readonly BlobServiceClient _blobServiceClient; 

    public ProcessExportJob(
        ILogger<ProcessExportJob> logger, 
        IExportRepository exportRepository,
        ITaskRepository taskRepository,
        BlobServiceClient blobServiceClient)
    {
        _logger = logger;
        _exportRepository = exportRepository;
        _taskRepository = taskRepository;
        _blobServiceClient = blobServiceClient;
    }

    [Function(nameof(ProcessExportJob))]
    public async Task Run([QueueTrigger("exports", Connection = "AzureStorage")] QueueMessage message)
    {
        _logger.LogInformation("Background execution triggered. Message ID: {MessageId}", message.MessageId);

        var payload = JsonSerializer.Deserialize<ExportQueueMessage>(message.MessageText);
        if (payload == null || string.IsNullOrEmpty(payload.JobId)) return;

        var job = await _exportRepository.GetExportJobAsync(payload.JobId);
        if (job == null) return;

        // Flip status to 'Processing'
        job.Status = "Processing";
        await _exportRepository.UpdateExportJobAsync(job); 

        try
        {
            _logger.LogInformation("Retrieving task dataset for Workspace {WorkspaceId}...", payload.WorkspaceId);

            // Query active tasks from Cosmos DB using TaskItem
            var tasks = await _taskRepository.GetTasksByWorkspaceAsync(payload.WorkspaceId);
            var tasksList = tasks?.ToList() ?? new List<TaskItem>();

            // Generate CSV payload with Excel Injection protection
            var csv = new StringBuilder();
            csv.AppendLine("TaskId,Title,Status,AssignedTo,Tags");

            foreach (var task in tasksList)
            {
                var formattedTags = string.Join("|", task.Tags ?? new List<string>());
                csv.AppendLine($"{task.Id},{EscapeCsv(task.Title)},{task.Status},{EscapeCsv(task.AssignedTo)},{EscapeCsv(formattedTags)}");
            }

            // Connect to private 'exports' container and stream
            var containerClient = _blobServiceClient.GetBlobContainerClient("exports");
            await containerClient.CreateIfNotExistsAsync();

            string targetBlobName = $"{payload.WorkspaceId}/{job.Id}.csv";
            var blobClient = containerClient.GetBlobClient(targetBlobName);

            using var memoryStream = new MemoryStream(Encoding.UTF8.GetBytes(csv.ToString()));
            await blobClient.UploadAsync(memoryStream, overwrite: true);

            // Mark job as completed and capture path
            job.Status = "Completed";
            job.BlobPath = targetBlobName;
            job.CompletedAt = DateTime.UtcNow;
            await _exportRepository.UpdateExportJobAsync(job);

            _logger.LogInformation("Job {JobId} completed successfully.", job.Id);
        }
        catch (Exception ex)
        {
            _logger.LogError(ex, "Uncaught exception inside background worker for message ID {MessageId}.", message.MessageId);
            job.Status = "Failed";
            job.ErrorMessage = ex.Message;
            await _exportRepository.UpdateExportJobAsync(job);
            throw; 
        }
    }

    private static string EscapeCsv(string? value)
    {
        if (string.IsNullOrEmpty(value)) return string.Empty;

        // Standard CSV quotes escaping
        if (value.Contains(",") || value.Contains("\"") || value.Contains("\n") || value.Contains("\r"))
        {
            value = $"\"{value.Replace("\"", "\"\"")}\"";
        }

        // Formula neutralization prefixing to protect Excel clients
        if (value.StartsWith("=") || value.StartsWith("+") || value.StartsWith("-") || value.StartsWith("@"))
        {
            value = $"'{value}";
        }

        return value;
    }
}

Step 2: User Delegation SAS & Browser Attachment Protection

A massive rookie mistake in cloud development is making storage containers public just so users can download files. Instead, our exports container is strictly private.

We generate a short-lived Shared Access Signature (SAS) URL that expires in 1 hour. Furthermore, instead of using legacy "Service SAS" (which relies on shared master storage account connection keys), we implemented User Delegation SAS. This signs the download URL dynamically using our Web API's own passwordless Microsoft Entra ID / Managed Identity!

In our production ExportController.cs, we implemented a critical Day 11 security upgrade: setting the ContentDisposition header. This instructs browsers to instantly download the file as a local attachment with a clean filename, rather than attempting to open and execute the raw CSV data inside the browser tab:

[HttpGet("download/{jobId}")]
public async Task<IActionResult> GetDownloadUrl(string jobId)
{
    _logger.LogInformation("Generating SAS download link for completed export job {JobId}.", jobId);

    var job = await _exportRepository.GetExportJobAsync(jobId);
    if (job == null) return NotFound();

    if (job.Status != "Completed" || string.IsNullOrEmpty(job.BlobPath))
    {
        return BadRequest("Export job is not completed or file path is missing.");
    }

    try
    {
        // Request a temporary User Delegation Key from Azure AD/Entra ID
        var options = new BlobGetUserDelegationKeyOptions(DateTimeOffset.UtcNow.AddDays(1))
        {
            StartsOn = DateTimeOffset.UtcNow
        };
        var userDelegationKeyResponse = await _blobServiceClient.GetUserDelegationKeyAsync(options);
        var userDelegationKey = userDelegationKeyResponse.Value;

        // Define read-only permissions and 1-hour expiration limits
        var sasBuilder = new BlobSasBuilder
        {
            BlobContainerName = "exports",
            BlobName = job.BlobPath,
            Resource = "b", // Targeting a single blob
            StartsOn = DateTimeOffset.UtcNow,
            ExpiresOn = DateTimeOffset.UtcNow.AddHours(1)
        };
        sasBuilder.SetPermissions(BlobSasPermissions.Read);

        // 🔥 Day 11 Security Upgrade: Force the browser to download as a secure file attachment
        sasBuilder.ContentDisposition = $"attachment; filename=\"tasks_export_{DateTime.UtcNow:yyyyMMdd}.csv\"";

        var containerClient = _blobServiceClient.GetBlobContainerClient("exports");
        var blobClient = containerClient.GetBlobClient(job.BlobPath);
        
        var sasUri = blobClient.GenerateUserDelegationSasUri(sasBuilder, userDelegationKey);

        _logger.LogInformation("Successfully generated secure User Delegation SAS URL for Job {JobId}.", jobId);
        return Ok(new { downloadUrl = sasUri.ToString() });
    }
    catch (Exception ex)
    {
        _logger.LogError(ex, "Failed to generate User Delegation SAS URL for completed Job {JobId}.", jobId);
        return StatusCode(500, $"Internal server error generating download URL: {ex.Message}");
    }
}

Developer Gotcha: The RBAC Permissions Hurdle

Critical Azure Configuration: To use User Delegation SAS successfully, the identity running your Web API needs more than just basic Storage Blob Data Reader credentials. You must explicitly assign your API's App Service/Identity the **Storage Blob Converter / Storage Blob Data Delegator** role in Azure. Without it, requesting the delegation key will trigger an immediate, unhandled 403 Forbidden error!

Step 3: Frontend Polling and Download Handshake

On our frontend, we designed a responsive, polling-based UX flow to handle this asynchronous, non-blocking export pattern:

Trigger: The user clicks the "Export Workspace" button, initiating a POST request to /api/Export/request/{workspaceId}.
Acceptance: The API immediately responds with a 202 Accepted status and a jobId, keeping the user interface completely unlocked.
Polling: The UI displays a progress spinner and queries /api/Export/status/{jobId} every 3 seconds.
Acquisition: Once the status updates to Completed, the frontend calls /api/Export/download/{jobId} to retrieve our short-lived SAS URL.
Download: The frontend dynamically injects a temporary hidden anchor (<a>) tag into the DOM, binds the SAS URL to the href attribute, clicks it programmatically to prompt the native browser download, and instantly cleans up the DOM.

Architecture Snapshot

Our complete asynchronous file generation architecture flows as follows:

┌──────────────────┐               1. Request Export            ┌──────────────────┐
│                  │ ─────────────────────────────────────────> │                  │
│                  │ <───────────────────────────────────────── │                  │
│                  │            2. Return 202 (JobId)           │                  │
│                  │                                            │     .NET API     │
│                  │             5. Poll Status /               │                  │
│                  │            Retrieve Short SAS URL          │                  │
│   Web Frontend   │ ─────────────────────────────────────────> │                  │
│                  │ <───────────────────────────────────────── │                  │
│                  │                                            └────────┬─────────┘
│                  │                                                     │
│                  │                                                     │ 3. Enqueue
│                  │                                                     ▼
│                  │                                            ┌──────────────────┐
│                  │                                            │  Storage Queue   │
│                  │                                            └────────┬─────────┘
│                  │                                                     │
│                  │                                                     │ 4. Trigger
│                  │                                                     ▼
│                  │             6. Download CSV File           ┌──────────────────┐
│                  │ <───────────────────────────────────────── │  Azure Function  │
│                  │                                            │   Queue Worker   │
└──────────────────┘                                            └────────┬─────────┘
         ▲                                                               │
         │                                                               │ 5. Stream
         └───────────────────────────────────────────────────────────────┘
                                Write Private File 
                               (Protected Blob Storage)

Day 11 Wrap-Up

By completing Day 11, we successfully achieved:

Real in-memory CSV compilation with strict, injection-safe parameter escaping.
Passwordless Blob integration streaming file buffers straight to protected cloud storage containers.
A secure download endpoint leveraging identity-delegated SAS tokens to guarantee corporate file security.
Resilient failure modes that record execution issues transparently inside Cosmos DB and Application Insights.

Our application is officially delivering true, high-fidelity business value to our users while maintaining an uncompromising, zero-trust cloud security posture!

Day 11: Export File Generation & Blob Storage Integration

Why Real-World File Generation Matters

Step 1: Extending the Function Worker with Dependency Injection

Step 2: User Delegation SAS & Browser Attachment Protection

Developer Gotcha: The RBAC Permissions Hurdle

Step 3: Frontend Polling and Download Handshake

Architecture Snapshot

Day 11 Wrap-Up

Comments

More from this blog

Day 10: Observability with Application Insights Telemetry

Day 9: Production Network Handshakes — CORS & Environment Variables

Day 8: Asynchronous Workers, Passwordless IaC, & Multi-App CI/CD

Day 7.5: Architectural Reconciliation: CORS, IaC Drift, and CI/CD Completeness

Command Palette

Why Real-World File Generation Matters

Step 1: Extending the Function Worker with Dependency Injection

Step 2: User Delegation SAS & Browser Attachment Protection

Developer Gotcha: The RBAC Permissions Hurdle

Step 3: Frontend Polling and Download Handshake

Architecture Snapshot

Day 11 Wrap-Up

Comments

More from this blog