adllm Insights logo adllm Insights logo

Taming GHC Plugin Dragons: Profiling and Fixing Performance Bottlenecks

Published on by The adllm Team. Last modified: . Tags: haskell ghc compiler-plugins performance profiling optimization

GHC compiler plugins are a double-edged sword. They offer unparalleled power to inspect and transform code during compilation, enabling custom optimizations, domain-specific languages, and advanced static analysis. However, when a plugin misbehaves in a large Haskell project, it can transform GHC into a sluggish beast, dragging build times from minutes to hours.

This article is your guide to taming these performance dragons. We’ll explore how to pinpoint plugin-induced slowdowns, profile the plugin’s own Haskell code as it runs within GHC, identify common bottleneck patterns, and apply targeted optimizations.

The Challenge: Plugins Inside the Beast

A GHC plugin is not an external tool; it’s a Haskell module loaded and executed by GHC itself. It operates on GHC’s internal data structures—Abstract Syntax Trees (ASTs), Core (GHC’s intermediate language), or type checker state. This deep integration is powerful but means a plugin’s inefficiencies directly impact GHC’s performance.

In large projects, with numerous modules and complex dependency graphs, even minor plugin inefficiencies are magnified, leading to:

  • Excessive CPU Time: The plugin might be performing computationally expensive traversals or algorithms.
  • High Memory Usage & GC Pressure: The plugin could be allocating large temporary structures or inadvertently retaining references to compiler data, starving GHC of memory.

Diagnosing these issues requires a multi-layered profiling approach.

Step 1: Is It Really the Plugin? Confirming the Culprit

Before diving into intricate plugin profiling, confirm the plugin is indeed the source of the slowdown.

Differential Profiling

Compile a representative module or a subset of your project under two conditions:

  1. With the plugin: ghc -fplugin=My.Plugin.Module ... MyModule.hs
  2. Without the plugin: ghc -fno-plugin=My.Plugin.Module ... MyModule.hs (or by removing the plugin from build options).

A significant difference in compilation time is a strong initial indicator.

GHC Timings

GHC can report time spent in its major passes.

1

ghc -ddump-timings MyModule.hs

Look at the output. If a pass associated with your plugin (e.g., “Core Lint after MyPluginPass” for a Core plugin) shows a disproportionate amount of time, it warrants investigation. Verbose output (-v3 or -v4) might also show plugin start/end messages.

Step 2: Profiling the Plugin’s Haskell Code

Once the plugin is implicated, you need to profile its own Haskell code. This involves two stages:

A. Compile Your Plugin Package for Profiling

Your plugin is a Haskell package. It must be built with profiling enabled.

Using Cabal:

1

cabal build my-plugin-package --enable-profiling --ghc-options="-fprof-auto -fprof-cafs"

Or using Stack:

1

stack build --profile --ghc-options="-fprof-auto -fprof-cafs" my-plugin-package

This instruments your plugin’s code with cost centres.

B. Compile Your Main Project with Profiling RTS Options

Now, when compiling the project that uses the plugin, instruct GHC (and thus the loaded plugin) to generate profiling data.

1

ghc -fplugin=My.Plugin.Module <other_options> MyProblematicModule.hs +RTS -p -hc -l -N -RTS

Key RTS options:

  • +RTS -p -RTS: Generates a time and allocation profiling report (MyProblematicModule.prof).
  • +RTS -hc -RTS: Profiles by cost-centre stack, giving more context to where time is spent (recommended).
  • +RTS -l -RTS: Generates an eventlog (MyProblematicModule.eventlog) for analyzing GC behavior and thread activity.
  • +RTS -N -RTS: Ensures the plugin (and GHC) can use multiple cores if applicable.

C. Analyze the Profiling Data

  • .prof File: This is your primary tool.

    • View with hp2ps -c MyProblematicModule.prof && ps2pdf MyProblematicModule.ps (then view PDF).
    • Use profiteur MyProblematicModule.prof for an interactive HTML report.
    • Use ghc-prof-flamegraph MyProblematicModule.prof to generate a flame graph. These reports will show which functions within your plugin package are consuming the most time and allocations.

  • .eventlog File:

    • Analyze with threadscope (older GUI tool) or ghc-events-analyze summary MyProblematicModule.eventlog (text-based) or eventlog2html MyProblematicModule.eventlog.
    • Look for excessive garbage collection (GC) activity or long pauses during the plugin’s execution phase.

Common Plugin Performance Anti-Patterns and Fixes

Profiling will point to where the time is spent. Why it’s spent often falls into these categories:

1. Inefficient AST/Core Traversal

  • Problem: Blindly using generic traversal libraries (like syb’s everywhere) for large ASTs, performing multiple full traversals where one would suffice, or using deeply recursive patterns inefficiently.
  • Fixes:
    • Targeted Recursion: Write specific recursive functions that understand the structure of HsExpr, CoreExpr, Pat, etc. Use GHC’s helper functions like collectArgs, collectBinders for Core.
    • Memoization: If plugin logic re-evaluates properties of shared sub-expressions, cache these results.
    • Pruning/Short-Circuiting: Design traversals to stop early if a condition is met or a subtree is irrelevant.
    • Understand GHC Data Structures: Familiarize yourself with GHC.Hs.* and GHC.Core.* to navigate them effectively.
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13

-- Conceptual: Inefficient vs. Efficient Traversal
-- Inefficient (e.g., using a generic SYB query everywhere)
-- listify (\(L _ (Var (Unqual occ))) -> isInteresting occ) hsModuleAst

-- More Efficient (targeted recursion for specific HsExpr constructors)
findInterestingVars :: LHsExpr GhcPs -> [OccName]
findInterestingVars (L _ (HsVar _ (L _ varName)))
  | isInteresting (occName varName) = [occName varName]
  | otherwise                       = []
findInterestingVars (L _ (HsApp _ fun arg)) =
  findInterestingVars fun ++ findInterestingVars arg
-- ... other HsExpr cases, potentially using a fold or monadic traversal
findInterestingVars _ = []

2. Excessive Allocation or Memory Retention

  • Problem: Creating numerous intermediate data structures, especially Strings; holding onto large GHC AST/Core fragments longer than necessary (preventing GHC from GC’ing them).
  • Fixes:
    • Strictness: Use strict fields (!) in plugin-internal data types. Force evaluation of intermediate results with seq or bang patterns if they are no longer needed.
    • Profile Allocations: Use +RTS -hy -RTS (by type) or +RTS -hc -RTS (by cost-centre stack) to identify sources of high allocation.
    • FastString: When interacting with GHC APIs that use FastString (GHC’s interned string type), work with FastStrings directly as much as possible to avoid costly unpackFS/mkFastString conversions.
    • Text / ByteString: For plugin-internal string processing, prefer these over String.
    • Scope Management: Ensure large GHC data structures fetched via plugin APIs (e.g., ModGuts) are not retained by your plugin’s state longer than a single module’s processing.

3. Redundant Work Across Modules or Invocations

  • Problem: If a plugin needs project-wide information (e.g., a summary of all exports), recomputing this for every module is extremely wasteful.
  • Fixes:
    • Plugin Arguments: Use -fplugin-opt My.Plugin:config-file=path/to/shared.cfg to pass pre-computed information.
    • Annotations (Advanced): A plugin can write data into module annotations, which can then be read by subsequent plugin invocations or by GHC itself. This is complex but powerful for caching.
    • TcPluginM / CoreM State: For state within a single module compilation, use the monadic state of TcPluginM or CoreM.

4. Slow Interaction with GHC APIs

  • Problem: Repeatedly calling GHC functions that perform complex lookups (e.g., looking up Names, TyCons, or class instances) without caching.
  • Fixes:
    • Caching: If your plugin repeatedly queries GHC for the same information within a pass, cache the results in the plugin’s local state.
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11

-- Conceptual: Caching GHC API calls
-- Inside your CoreM or TcPluginM monad:
cachedLookup :: Name -> MyPluginM TyCon
cachedLookup name = do
  cache <- getMyPluginCache -- from plugin's state
  case Map.lookup name cache of
    Just tyCon -> return tyCon
    Nothing    -> do
      tyCon <- liftIO $ GHC.lookupTyCon name -- Simplified GHC API call
      setMyPluginCache (Map.insert name tyCon cache)
      return tyCon

5. Type Checker Plugin Inefficiencies

  • Problem: Solver plugins generating an excessive number of complex constraints, or providing evidence inefficiently, can bog down GHC’s type checker.
  • Fixes: This is highly specialized. Focus on simplifying the constraints your plugin emits or making evidence generation more direct. Consult GHC’s documentation on writing type-checker plugins.

Fine-Grained Debugging and Tracing

Sometimes, standard profiling isn’t enough.

  • Debug.Trace.traceMarkerIO / traceEventIO: Insert these around specific sections of your plugin code:
    1
2
3
4
5

    import Debug.Trace (traceMarkerIO)
-- ...
liftIO $ traceMarkerIO "MyPlugin: Starting complex analysis"
-- ... perform complex analysis ...
liftIO $ traceMarkerIO "MyPlugin: Finished complex analysis"
    These markers will appear in the eventlog, allowing you to correlate plugin phases with GC activity or overall timeline.
  • Custom Timers: For critical sections, use System.CPUTime.getCPUTime or Data.Time.Clock.getCurrentTime to print execution times to stderr (guarded by a debug flag passed via -fplugin-opt).

Key GHC Flags for Plugin Development & Profiling

  • Plugin Control: -fplugin=M, -fno-plugin=M, -fplugin-opt=M:args
  • Profiling: --enable-profiling (for plugin package), +RTS -p -hc -l -N -RTS (for main project)
  • Diagnostics: -ddump-timings, -v3 (or -v4), -ddump-parsed-ast, -ddump-rn-ast, -ddump-core, -ddump-to-file

Conclusion

Optimizing GHC compiler plugin performance is an iterative process: identify the general slowdown, confirm the plugin’s role, profile the plugin’s internals, analyze the results, and apply targeted fixes. While challenging, the reward is significant: faster builds, a more responsive development cycle, and the ability to leverage GHC’s extensibility without undue performance penalties. By understanding common pitfalls and mastering Haskell’s powerful profiling tools, you can ensure your GHC plugins are assets, not anchors, to your large Haskell projects.