Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
27 changes: 27 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -101,6 +101,33 @@ It's also convenient for profiling, debugging, etc, especially since all benchma
ruby benchmarks/some_benchmark.rb
```

### Benchmark organization

Benchmarks can be organized in three ways:

1. **Standalone .rb files** - Place a `.rb` file directly in the `benchmarks/` directory:
```
benchmarks/fib.rb # Benchmark name: "fib"
```

2. **Single benchmark per directory** - For benchmarks that need additional files (like Gemfiles):
```
benchmarks/erubi/
benchmark.rb # Benchmark name: "erubi"
Gemfile
```

3. **Multiple benchmarks per directory** - For related benchmarks sharing dependencies:
```
benchmarks/addressable/
equality.rb # Benchmark name: "addressable-equality"
join.rb # Benchmark name: "addressable-join"
Gemfile # Shared Gemfile
```

In directories **without** a `benchmark.rb` file, all `.rb` files will be discovered as separate benchmarks.
The benchmark name is derived as `directoryname-suffix` from `suffix.rb` files.

## Ractor Benchmarks

ruby-bench supports Ractor-specific benchmarking with dedicated categories and benchmark directories.
Expand Down
5 changes: 0 additions & 5 deletions benchmarks/addressable-getters/Gemfile

This file was deleted.

16 changes: 0 additions & 16 deletions benchmarks/addressable-getters/Gemfile.lock

This file was deleted.

5 changes: 0 additions & 5 deletions benchmarks/addressable-join/Gemfile

This file was deleted.

16 changes: 0 additions & 16 deletions benchmarks/addressable-join/Gemfile.lock

This file was deleted.

5 changes: 0 additions & 5 deletions benchmarks/addressable-merge/Gemfile

This file was deleted.

16 changes: 0 additions & 16 deletions benchmarks/addressable-merge/Gemfile.lock

This file was deleted.

5 changes: 0 additions & 5 deletions benchmarks/addressable-new/Gemfile

This file was deleted.

16 changes: 0 additions & 16 deletions benchmarks/addressable-new/Gemfile.lock

This file was deleted.

5 changes: 0 additions & 5 deletions benchmarks/addressable-normalize/Gemfile

This file was deleted.

16 changes: 0 additions & 16 deletions benchmarks/addressable-normalize/Gemfile.lock

This file was deleted.

5 changes: 0 additions & 5 deletions benchmarks/addressable-parse/Gemfile

This file was deleted.

16 changes: 0 additions & 16 deletions benchmarks/addressable-parse/Gemfile.lock

This file was deleted.

5 changes: 0 additions & 5 deletions benchmarks/addressable-setters/Gemfile

This file was deleted.

16 changes: 0 additions & 16 deletions benchmarks/addressable-setters/Gemfile.lock

This file was deleted.

5 changes: 0 additions & 5 deletions benchmarks/addressable-to-s/Gemfile

This file was deleted.

16 changes: 0 additions & 16 deletions benchmarks/addressable-to-s/Gemfile.lock

This file was deleted.

File renamed without changes.
File renamed without changes.
85 changes: 85 additions & 0 deletions lib/benchmark_discovery.rb
Original file line number Diff line number Diff line change
@@ -0,0 +1,85 @@
# frozen_string_literal: true

# BenchmarkDiscovery handles discovering benchmarks in different organization patterns:
# 1. Standalone .rb files in benchmarks/ directory
# 2. Single benchmark.rb in a subdirectory
# 3. Multiple .rb files in a subdirectory (named as directory-suffix, excluding benchmark.rb)
class BenchmarkDiscovery
# Represents a discovered benchmark
BenchmarkEntry = Struct.new(:name, :script_path, :directory, keyword_init: true)

attr_reader :base_dir

def initialize(base_dir)
@base_dir = base_dir
end

# Returns an array of BenchmarkEntry objects
def discover
return [] unless Dir.exist?(base_dir)

entries = []

Dir.children(base_dir).sort.each do |entry|
entry_path = File.join(base_dir, entry)

if File.file?(entry_path) && entry.end_with?('.rb')
# Pattern 1: Standalone .rb file
entries << BenchmarkEntry.new(
name: entry.delete_suffix('.rb'),
script_path: entry_path,
directory: nil
)
elsif File.directory?(entry_path)
# Check for patterns 2 and 3
entries.concat(discover_directory_benchmarks(entry, entry_path))
end
end

entries
end

private

def discover_directory_benchmarks(dir_name, dir_path)
benchmark_files = find_benchmark_files(dir_path)
return [] if benchmark_files.empty?

entries = benchmark_files.map do |file|
create_benchmark_entry_in_directory(dir_name, dir_path, file)
end

entries.sort_by(&:name)
end

def find_benchmark_files(dir_path)
all_rb_files = Dir.children(dir_path).select { |file| file.end_with?('.rb') }

# If benchmark.rb exists, only use that (Pattern 2)
if all_rb_files.include?('benchmark.rb')
['benchmark.rb']
else
# Otherwise, use all .rb files (Pattern 3)
all_rb_files
end
end

def create_benchmark_entry_in_directory(dir_name, dir_path, file)
if file == 'benchmark.rb'
# Pattern 2: Single benchmark.rb in directory
BenchmarkEntry.new(
name: dir_name,
script_path: File.join(dir_path, file),
directory: dir_name
)
else
# Pattern 3: Multiple .rb files (derive suffix from filename without .rb extension)
suffix = file.delete_suffix('.rb')
BenchmarkEntry.new(
name: "#{dir_name}-#{suffix}",
script_path: File.join(dir_path, file),
directory: dir_name
)
end
end
end
28 changes: 24 additions & 4 deletions lib/benchmark_filter.rb
Original file line number Diff line number Diff line change
Expand Up @@ -2,16 +2,16 @@

# Filters benchmarks based on categories and name patterns
class BenchmarkFilter
def initialize(categories:, name_filters:, excludes:, metadata:)
def initialize(categories:, name_filters:, excludes:, metadata:, directory_map: {})
@categories = categories
@name_filters = process_name_filters(name_filters)
@excludes = excludes
@metadata = metadata
@category_cache = {}
@directory_map = directory_map
end

def match?(entry)
name = entry.sub(/\.rb\z/, '')
def match?(name)
matches_category?(name) && matches_name_filter?(name) && !matches_excludes?(name)
end

Expand All @@ -27,7 +27,27 @@ def matches_category?(name)
def matches_name_filter?(name)
return true if @name_filters.empty?

@name_filters.any? { |filter| filter === name }
@name_filters.any? do |filter|
if filter.is_a?(Regexp)
filter === name
else
# Exact match
next true if filter == name

matches_prefix_in_same_directory?(name, filter)
end
end
end

# Prefix match only for benchmarks in the same directory
# e.g., "addressable" matches "addressable-equality" if they're in the same dir
# but "erubi" does NOT match "erubi-rails" if they're in different dirs
def matches_prefix_in_same_directory?(name, filter)
return false unless name.start_with?("#{filter}-")

benchmark_dir = @directory_map[name]
# Only match if the benchmark is in a directory with the filter name
benchmark_dir == filter
end

def matches_excludes?(name)
Expand Down
Loading
Loading