All users being able to complete tasks via REST

I have a task that has the assignee set to user1. But, all users in the system are being able to complete the task via REST. I also have authorization enabled and working. How can I make sure that only the assigned user is able to complete the task?