If you want to fully delete a membership, you can do it under the three dots next to the project membership. If you want to delete multiple memberships at once, you can multi-select all those memberships and then click on the three dots above the listing to delete all of those.
Deletion is permanent and it will remove everything related to that membership. If you want to save the data on the membership (for example past allocations and such), you should archive the membership instead of deleting it.
Deletion should only be used if the membership was added by accident to the project or similar scenarios.