Benchmarked new perl class feature with many class builders[2024-06-10 updated]

[Updated]

• (2024-06-10 11:13 JST) update to v5.40 benchmarked result GitHub - kfly8/bench-perl-class-builder at v5.40
• (2023-08-22 21:30 JST) Added the following pull requests, then benchmarked. Thanks @bbrtj
  • Add XS variants for Moose and Moo
• (2023-08-22 15:30 JST) Added the following pull requests, then benchmarked. Thanks @bbrtj
  • Add immutable for Moose class #1
  • Add two extra fields for each class #2

What's this?

In Perl 5.38, the class feature was integrated into the core, so I compared it with many class builders. Ovid, who is main designer of new class feature, blogged the follwing thing:

Note that it’s not taking anything away from Perl; it’s adding a core object system for better memory consumption, performance, and elegance.
( https://ovid.github.io/articles/corinna-in-the-perl-core.html )

The summary is as follows. Note that this class syntax is still experimental, so results may vary depending on future development.

• As of v5.38 [https://github.com/kfly8/bench-perl-class-builder/tree/v5.38:title]
  • This new class syntax was the most memory efficient, and the performance of constructors and object accessors was comparable to that of the array references with bles. (Performance was better than bless with the most used hash reference.)
• For v5.40 [https://github.com/kfly8/bench-perl-class-builder/tree/v5.40:title]
  • This new class syntax was the most memory efficient.
  • The constructor had performance comparable to that of the array reference blessed.
  • The object accessors performed as well as a hash reference without.

Execution Environment

 ❯ inxi -SCm
  System:
    Host: kfly8.local Kernel: 23.1.0 arch: arm64 bits: 64 Console: s006 OS: Darwin 23.1.0
  Memory:
    System RAM: total: N/A available: N/A used: N/A
    RAM Report: missing: Required tool dmidecode not installed. Check --recommends
  CPU:
    Info: 10-core model: Apple M2 Pro bits: 64 type: MCP
    Speed: N/A min/max: N/A cores: No OS support for core speeds.

Benchmark memory size

Size	Compare	Title
135.8 KB	--	class feature (perl: 5.040000)
257.9 KB	89.90%	bless arrayref
265.7 KB	95.60%	Object::Pad@0.808
359.5 KB	164.80%	Mouse@v2.5.10
359.5 KB	164.80%	Moo@2.005005
359.5 KB	164.80%	Moose@2.2207 (XSAccessor)
359.5 KB	164.80%	Object::Tiny@1.09
359.5 KB	164.80%	Moo@2.005005 (XSConstructor + XSAccessor)
359.5 KB	164.80%	Class::Tiny@1.008
359.5 KB	164.80%	Moose@2.2207
359.5 KB	164.80%	bless hashref
359.5 KB	164.80%	Class::Accessor::Lite@0.08

This result were calculated using bench-size.pl.

Benchmark object constructors

Rate	Compare	Title
550/s	-59%	Class::Tiny@1.008
752/s	-44%	Moose@2.2207
752/s	-44%	Moose@2.2207 (XSAccessor)
931/s	-30%	Moo@2.005005
1081/s	-19%	Moo@2.005005 (XSConstructor + XSAccessor)
1152/s	-14%	Mouse@v2.5.10
1195/s	-10%	Object::Pad@0.808
1334/s	--	class feature (perl: 5.040000)
1347/s	1%	bless arrayref
1674/s	25%	bless hashref
1690/s	27%	Class::Accessor::Lite@0.08
1729/s	30%	Object::Tiny@1.09

This result were calculated using bench-new.pl.

Benchmark access to object fields

Rate	Compare	Title
93699/s	-26%	Object::Pad@0.808
100485/s	-20%	Class::Accessor::Lite@0.08
120302/s	-5%	Moo@2.005005
123675/s	-2%	Class::Tiny@1.008
124842/s	-1%	Moose@2.2207
126030/s	--	class feature (perl: 5.040000)
127242/s	1%	bless hashref
137846/s	9%	bless arrayref
156038/s	24%	Object::Tiny@1.09
223418/s	77%	Moose@2.2207 (XSAccessor)
236307/s	87%	Mouse@v2.5.10
267962/s	113%	Moo@2.005005 (XSConstructor + XSAccessor)

This result were calculated using bench-field.pl.

Repository

https://github.com/kfly8/bench-perl-class-builder

Comments

[bbrtj]

Thanks for the benchmarks. Your Moose case for constructor is flawed, as you didn't make the class meta immutable: metacpan.org/dist/Moose/view/lib/M...

Also, I would suggest having at least a couple of fields in your class. Having just one may favor some simpler solutions which may not scale well. In my own benchmarks it seemed like Moo constructor did not scale as good as Moose one, while being faster for super simple cases.

[kobaken]

Thanks for your comment!

I agree that Moose (and also Mouse) would be better if they were immutable.
And you are right, I think the benchmark would be better if it handled cases where the fields are complex.

I would be happy to send me a pull request!

[kobaken]

Thanks for your pull requests! I updated this post and repository.

[Paul Evans]

Something to keep in mind with the field access benchmarks is that the very design of Object::Pad and the core class system offers direct access to the fields of the object to the class's own methods, as simple lexicals. Code inside methods does not have to use the accessors just to see or modify those values. This is a feature unique to those two systems.

Therefore, in real-world code scenarios, the faster more-direct access these provide means that overall code can run faster in a way that simple benchmarks "from outside the class" such as your bench-field.pl do not manage to capture. It would not be easily possible to write a "universal" test case in the same style as your scripts to demonstrate this, exactly because of this unique ability.

As an example, consider this silly class:

class RGBColour {
  field $red :param;
  field $green :param;
  field $blue :param;

  method as_css () {
    sprintf "rgb(%f%%,%f%%,%f%%)", $red, $green, $blue;
  }
}

The ->as_css method here has direct access to the $red, $green and $blue lexicals, and so invoking that would only involve one actual method call at runtime. Whereas, this class based on any of the other systems would need code something like:

sub as_css ($self) {
  sprintf ..., $self->red, $self->green, $self->blue;
}

and thus any call to the outer method would involve three inner calls, a total of four. Once you get into real-world scenarios with code that contains real-world behaviour, the cost of all these little inner $self->field accessor calls quickly adds up and overtakes the original cost of the outside method call or even the object constructor.

Perhaps you could add a new category of benchmark, one that constructs an object with a few nontrivial fields and then performs a calculation method based on the values of those fields - feel free to steal my example code above if it helps. I'd be very interested to see benchmarks of that kind of case too.

[kobaken]

Thanks for your kind comments!

I will try the new benchmark case!

[Dimitrios Kechagias]

It's interesting to see Mouse still leading the pack of the Moose compatible crowd. We switched to it a few years ago after benchmarking, and we've kept it even though Moo became more popular. It even manages to pull a win over the current version of class, but the latter is sure looking good otherwise, especially for something newly added to core!

[kobaken]

As you say, by far the best performing accessor was Mouse. I think the difference is whether the accessor works with XS or not. class feature will be adding a :reader modifier in the future that will make the accessor live, and I will try to benchmark it again when that is added.

[Bruno Meneguele]

Hey, really nicely done! Hope we can keep this benchmark alive as long as possible!

One suggestion: it would be really nice to keep a "history" graph to follow core class enhancements in future Perl versions. AFAIK only the MVP was introduced to the core, but many optimizations are still to be done, including (as you mentioned in one of your comments) :reader and :writer accessors and, possibly, many other low-level object handling optimizations. So I really expect to see core class feature numbers to decrease, but who knows right? :D

[kobaken]

Thank you for your comment!

You are right, it would be nice to keep a history! I look forward to following future Perl updates.

[Ed Sabol]

Very nice!

[chrisarg]

Thanks for the update.
Seems the implementation is catching up to blessed hash references.
What about inside out objects classes such as Class:Std, DIOS etc ?

[kobaken]

Thank you for your comment! I would be happy if you could send me a pull request!