Bukkit

These results are for v0.3 of SuperPermsTest on 1. Dec. 2011. To the newest version. To the previous version

Test Results

This page will display test results of myself running the testcases for the various plugins. You can use them as a reference to compare plugin performance. Be aware that the results will slightly differ between each run of the test (+- 5% usually), therefore small differences in runtime may have just been "bad luck". I ran the tests multiple times for each plugin and picked one of the most favourable for each.

The tests were set up according to this guideline and run on a fresh craftbukkit server, with a newly generated world and no other plugins besides SuperPermsTest and the relevant permissions plugin (and support plugins,usually a fake "Permissions" plugin).

How to read the tables:

You'll see two tables. The first table displays results collected by calling "player.hasPermission(node);", which is the usual way of checking permissions against "SuperPerms". The second displays results collected by calling "permissionHandler.has(player, world, node)", which is the "old" way to interact with plugins that are Permissions 2/3 compatible.

Each table has some columns:

• The first column will show the testcase. To find out what each testcase is about, read this.
• The second column the checked "node".
• The next five columns show the result we get from bP (bPermissions 1.9.1), GM (Essentials GroupManager 1.6.18 Phoenix), PB (PermissionsBukkit 1.2) and PEx (PermissionsEx 1.17).
• The last columns will show the time needed for a lookup in ns (only if the testcase was a success). Smaller times are usually better.

If a plugin failed a part of a testcase, the result is written in red font. This means that the plugins results didn't match any of the expected results for that testcase (there may be more than one accepted result for a testcase, so don't wonder if plugins pass a test despite having different results).

All tests were done with a file backend.

Test results for SuperPerms

Using default sample size of 10000

Test results for "old" Permissions (bridges/compat layers etc.)

Using default sample size of 10000

Interpreting the results

Here are my thoughts about the results that you can see above. They are by no means the be all, end all of truth, just educated guesses and ideas.

• SuperPerms Correctness: All plugins manage to deliver expected results, although some may differ in how they interpret permission nodes that contain a "*". PermissionsEx fixed the problem that would sometimes prevent SuperPerms from working correctly.

• SuperPerms Speed: All plugins are very fast in their latest versions. PermissionsEx 1.17 manages to be faster in checking permissions that evaluate to "false" than the rest.

• "Old" Permissions Correctness: Again, all plugins manage to deliver expected results. Either they deliver "SuperPerms" results like bPermissions and PermissionsBukkit), or results that are similar to what Permissions 3 would give. If you plan on using the Permissions 3 API in your plugin to query permissions, be aware of those differences!

• "Old" Permissions Speed: All plugins manage to be very fast in their latest versions, except for PermissionsBukkit, which falls a bit behind. PermissionsEx 1.17 manages to be a bit faster than the others. Essentials GroupManager fixed the serious performance problems it had before.

Differences between the plugins

I just want to make a small list of key differences/similarities to help people understand why something may (not) work with specific plugins. If you have ideas on what other properties should be listed in this table, message me. This list should be about how the plugins are designed, not if they have (currently) a bug or similar.

(*)GroupManager ignores permission attachments in these cases, but respects default permission settings.