A design change suggestion would be to have Job Groups to which we add jobs and people. That way the control is from the group but access to any job under that group can be easily changed along with the people.
I do not know of a way, unless you make a new group with specific access that covers your specific need. But I am sure there is overlap with other jobs you want people to have or not have access to. Permissions are a pain if you have a lot of users.
I think to do it through powershell or the API you would need to know the ID of the job(s) and the specific user permissions you need along with the user ID's. I would be surprised if anyone has a unique, easy or intuitive way to do this.