SuperPermsTest

Table of Contents

This project is abandoned and its default file will likely not work with the most recent version of Minecraft. Whether this project is out of date or its author has marked it as abandoned, this project is no longer maintained.

SuperPermsTest

SuperPermsTest allows to test permission plugins that claim to support "SuperPerms" for their correctness and performance.

Introduction

Bukkit created a standard API for setting and retrieving permissions of players called "SuperPerms" and there are currently a handful of permissions plugins that claim to support that API. This plugin runs testcases (see the Test Details page) against such a plugin on the server, trying to spot faulty implementations, misconfigurations or performance problems with "SuperPerms".

For Server Admins

If you are just interested in the results of the testcases, head over to the Test Results section and take a look at them. Those results are from my home PC (4 core AMD 2.8 GHz, 4 GB Ram) and while your server will likely produce different times, the relation between the times should stay roughly the same.

If you want to run these tests yourself against the permission plugin you currently use on your server, you should go to the Test Setup section and read about what you'll have to do to successfully run these testcases (you'll have to choose a player for testing and give him some permission nodes). You may spot performance or configuration problems on your server that way.

For Plugin Developers

If your plugin uses "SuperPerms", you may want to take a look at the Test Results section. It will show you some expected times for how long a call to "player.hasPermission(...)" may take under certain circumstances. You can also setup the test plugin on your test server by following instructions on the Test Setup page to make sure that the permissions plugin you use for testing your plugin works correctly and therefore can be sure it is (not) your fault if something doesn't work.

For Permissions Plugin Developers

If you plan on supporting "SuperPerms" or to make drastic changes to your implementation, running the testcases provided by "SuperPermsTest" can help you to spot errors early, instead of waiting for people to give you obscure error reports like "it doesn't work. Fix it!". Also, you may use the test results on the Test Results page to convince people that your plugin works correctly, is fast enough etc. and that it has to be an user error that's causing the problems (e.g. misconfiguration).

Some other info

I don't claim that these testcases are complete, meaning that a plugin passing all testcases is not equal to it implementing SuperPerms 100%. But it means that most of the practical usecases of "SuperPerms" are covered.

The project is released under MIT license. It is open source and you can find it on github here: SuperPermsTest.

I have a twitter account, which I use to announce new versions of my plugins (mostly):

Comments

_ForgeUser6899237
- Join Date: 5/26/2011
- Posts: 19
- Member Details
#31 _ForgeUser6899237

View User Profile

Send Message

Posted Mar 21, 2012
@Evenprime85

Requesting new stats please! All of the perms plugins have updated since your last test and it would be really great if we had some new stats to ogle.

Rollback Post to Revision RollBack
_ForgeUser6934774
- Join Date: 5/31/2011
- Posts: 8
- Member Details
#30 _ForgeUser6934774

View User Profile

Send Message

Posted Jan 26, 2012
@Evenprime85 I really appreciate the work you're doing with permissions. It's a area of Bukkit I feel very strongly about. Even today there's still plugins with horrible Permission resolvers etc because they still want to support outdated systems. I'm glad you're doing you're best to improve things, and keep up the good work.

Sam

Rollback Post to Revision RollBack
Forge_User_19096671
- Join Date: 7/22/2011
- Posts: 83
- Member Details
#29 Forge_User_19096671

View User Profile

Send Message

Posted Dec 17, 2011
I'd like to see a little colour coding to the speed replies, maybe >200ns = orange. >400ns = red. Make it more obvious which results are returning slowly.

Rollback Post to Revision RollBack
_ForgeUser7459264
- Join Date: 8/28/2011
- Posts: 371
- Member Details
#28 _ForgeUser7459264

View User Profile

Send Message

Posted Dec 1, 2011
Finally got around to make a new testrun with all the new versions of the plugins.

1. Good news: PermissionsEx fixed the random problem of not working SuperPerms in v1.17 2. Good news: GroupManager fixed the performance problems when using the "old" permissions API 3. Good news: All plugins behave now identically in all situations of SuperPerms, except for those cases where a "*" is involved in the permission name.

All in all I'm impressed by how much how much the overall situation has improved.

Rollback Post to Revision RollBack
Sleakes
- Location: Portland
- Join Date: 9/30/2007
- Posts: 122
- Member Details
#27 Sleakes

View User Profile

Send Message

Posted Nov 28, 2011
@Evenprime85

Towny uses groups/prefix/suffix etc heavily which could be easily replicated by just using Vault, but that's their issue I guess.

I guess it's more of a frustration than anything, as people try to tell me my plugin is broken when no... It's the permission handler that can't check properly, or go tell the other plugin to update their permission checks :p

Last edited by Sleakes: Nov 28, 2011

Rollback Post to Revision RollBack
_ForgeUser7459264
- Join Date: 8/28/2011
- Posts: 371
- Member Details
#26 _ForgeUser7459264

View User Profile

Send Message

Posted Nov 28, 2011
@Sleaker

That's the problem for me. The bridge gives people the "Permissions 3.1.6" behaviour, an that is oblivious to anything that is related to SuperPerms. I don't see how I could call that behaviour "wrong", as there is no common definition of "right" in that case.

The whole problem would naturally go away if plugin developers would just stop supporting "Permissions 3.1.6" altogether, as there is really no reason left for supporting it. I don't do it.

Rollback Post to Revision RollBack
Sleakes
- Location: Portland
- Join Date: 9/30/2007
- Posts: 122
- Member Details
#25 Sleakes

View User Profile

Send Message

Posted Nov 28, 2011
@Evenprime85

Well, my assumption is that if the permission plugin provides a bridge, it should work properly with all checks, or not provide a bridge. Including a bridge, failing one of the tests, then saying it's not a bug of the bridge is silly, regardless of where the permission adjustments are coming from (internal to the permission plugin, or other plugins) a bridge should be able to see these regardless (bPerms does it just fine and maintains good speed).

And that's not to mention GM's horrible offline/bridged checks.

Last edited by Sleakes: Nov 28, 2011

Rollback Post to Revision RollBack
_ForgeUser7459264
- Join Date: 8/28/2011
- Posts: 371
- Member Details
#24 _ForgeUser7459264

View User Profile

Send Message

Posted Nov 28, 2011
@Sleaker

Isn't that the fault of towny then, if it prefers inquiring a legacy permissions bridge instead of the actual SuperPerms API?

Rollback Post to Revision RollBack
Sleakes
- Location: Portland
- Join Date: 9/30/2007
- Posts: 122
- Member Details
#23 Sleakes

View User Profile

Send Message

Posted Nov 28, 2011
The results for the 2 new tests aren't color-coded properly for failed tests. (test 13/14) on the Bridged sections. Some say true some say false, when the expected result should be True.

The reason why I really wanted people to be aware of these is because plugins like Towny will super-fail when using PEX/GM and a seperate Super-Perms compatible plugin (like heroes) which adds an attachment onto a player to give them permissions for Towny (which checks the bridge API when it's detected), or if the plugin is unknowingly using case-sensetive checks onto lower-cased permissions (since everything is lowered in bukkit).

And it's pretty evident at the moment that bPerms is the solid permission choice, as it passes all the tests and gives good performance in all areas.

Last edited by Sleakes: Nov 28, 2011

Rollback Post to Revision RollBack
_ForgeUser7459264
- Join Date: 8/28/2011
- Posts: 371
- Member Details
#21 _ForgeUser7459264

View User Profile

Send Message

Posted Nov 23, 2011
@essentialsteam

Thanks for clarifying that. :)

Rollback Post to Revision RollBack
_ForgeUser7442447
- Join Date: 8/25/2011
- Posts: 35
- Member Details
#20 _ForgeUser7442447

View User Profile

Send Message

Posted Nov 23, 2011
Both GM build server belong to Essentials, they should have the same builds, since they are build automatically. Those are development builds. The official release is always with a new Essentials release.

The Bridge is still needed for plugins that require the old Permissions interface.

Rollback Post to Revision RollBack
_ForgeUser7459264
- Join Date: 8/28/2011
- Posts: 371
- Member Details
#19 _ForgeUser7459264

View User Profile

Send Message

Posted Nov 23, 2011
PermissionsEx 1.17 will fix the error that causes SuperPerms to fail to work correctly sometimes, if the plugins are loaded in the "wrong" order.

When that version gets available "officially", I'll run a new round of tests for it.

Rollback Post to Revision RollBack
_ForgeUser7459264
- Join Date: 8/28/2011
- Posts: 371
- Member Details
#18 _ForgeUser7459264

View User Profile

Send Message

Posted Nov 22, 2011
@LlmDl

Hm, I tried to run with 1.6.2 GroupManager (seems both sites have the same versions after all) and without the Bridge plugin, but in that case I can't get a reference to the (fake) "Permissions" plugin that's necessary to do the old style of permissions checking.

Plugin p = getServer().getPluginManager().getPlugin("Permissions"); if(p != null) pHandler = ((Permissions) p).getHandler();

Will leave me with pHandler = null; Beyond that, the results are identical to the version I tested. Maybe the "shouldn't be needed anymore" is/was a planned feature or refers to all important plugins supporting SuperPerms anyway now?

Have you tried to run it without the bridge and a plugin that only uses old permissions?

PS: Also tested to do the above during runtime of the server to be sure that GM is fully started. Still, getServer().getPluginManager().getPlugin("Permissions"); will return "null";
Last edited by _ForgeUser7459264: Nov 22, 2011

Rollback Post to Revision RollBack
_ForgeUser6899237
- Join Date: 5/26/2011
- Posts: 19
- Member Details
#17 _ForgeUser6899237

View User Profile

Send Message

Posted Nov 22, 2011
@Evenprime85

I'm not certain, you'd better use the ess.ementalo.com. Elgarl, lead dev of towny, picked up GM after the ess team pretty much dropped support for it.

Rollback Post to Revision RollBack
_ForgeUser7459264
- Join Date: 8/28/2011
- Posts: 371
- Member Details
#16 _ForgeUser7459264

View User Profile

Send Message

Posted Nov 21, 2011
@LlmDl

Will do.

Will the builds on http://ess.ementalo.com (GroupManager) be identical to those on http://ci.earth2me.net (Essentials)?

Rollback Post to Revision RollBack
_ForgeUser6899237
- Join Date: 5/26/2011
- Posts: 19
- Member Details
#15 _ForgeUser6899237

View User Profile

Send Message

Posted Nov 21, 2011
This is really cool but I'd like to see you test the newest GM. On another note, the groupmanager bridge is no longer needed, and GM supports old perms and superperms natively.

http:dev.bukkit.org/server-mods/groupmanager/

Last edited by _ForgeUser6899237: Nov 21, 2011

Rollback Post to Revision RollBack
_ForgeUser7459264
- Join Date: 8/28/2011
- Posts: 371
- Member Details
#14 _ForgeUser7459264

View User Profile

Send Message

Posted Nov 21, 2011
Woohoo, it's done. Well, that took longer than I wanted it to. Take a look at the completely redesigned Results page. I believe it is now much more interesting to read and to compare single values between plugins.

I also updated the "interpretation" part at the results page (be sure to read that) and test details.

Rollback Post to Revision RollBack
_ForgeUser7459264
- Join Date: 8/28/2011
- Posts: 371
- Member Details
#13 _ForgeUser7459264

View User Profile

Send Message

Posted Nov 21, 2011
I've finished the new version (just uploaded it, may take a bit to get visible here). Currently composing test results of the new version.

Biggest problem I ran into is what's posted one comment below.

The plugin now may accept different answers to tests, and may comment on the answers (e.g. warning that it's not a fully correct answer, but at least acceptable).

Also, "old" permissions are tested too now. Here I accept the behaviour of the old Permissions 2/3 plugins and the behaviour of SuperPerms as valid answers.

and some more small things. Need to update descriptions etc. now.

Rollback Post to Revision RollBack
_ForgeUser7459264
- Join Date: 8/28/2011
- Posts: 371
- Member Details
#12 _ForgeUser7459264

View User Profile

Send Message

Posted Nov 21, 2011
Ok, THIS was unexpected. I ran into a problem while finishing and testing the new version. It's a problem as obscure as it can get and may be the reason why many people see their plugins permissions fail despite doing everything correct.

https://github.com/t3hk0d3/PermissionsEx/issues/134

It seems plugin load order influences what you get from PermissionsEx, heavily.

Rollback Post to Revision RollBack
_ForgeUser7459264
- Join Date: 8/28/2011
- Posts: 371
- Member Details
#11 _ForgeUser7459264

View User Profile

Send Message

Posted Nov 20, 2011
@Sleaker

Just found out that I'll have time to work on this tomorrow (Monday). So I'll just include what I've got until then.

Also I'll change how I present the results. Instead of having 1 section per plugin I want to make just a list of the tests with 4 columns (one column for each plugin) to make it easier to spot differences.

Rollback Post to Revision RollBack
To post a comment, please login or register a new account.

Project ID

33699
Created

Nov 10, 2011
Last Released File

Nov 21, 2011
Total Downloads

1,293
License

MIT License

Bukkit

SuperPermsTest

Bukkit Plugins

SuperPermsTest

Table of Contents

SuperPermsTest

Introduction

For Server Admins

For Plugin Developers

For Permissions Plugin Developers

Some other info

Comments

About This Project

Categories

Members

Recent Files

Bukkit