On one side, I personally think that immigration should never be an issue in any modern country. It is a part of the human nature, just like marriage and having a family. People should freely travel and choose which countries they want to settle in freely.
On the other hand, I think Brexit would actually impact it unless a quota on the EU immigrants is included in the deal he got from the EU. (To be honest I couldnt examine the deal in detail but as far as I know there is no such thing).
Why? because if the EU citizens are treated as third country nationals today, subject to the injust scrutiny of the home office, then the home office can simply and easily refuse millions of people. That is what they have been doing to third country nationals since removing the appeal rights to visa decisions. The audi alterem principle does not work for immigrants at this stage and such human rights breaches, noting that they are new, has not been challenged in the courts. Though I heard that there are some law firms trying to form a large group of victims and start a case.
Just to be more specific about what they do to current third country citizens who wants to apply for residence permits, work permits and even for visitor visas:
They refuse you saying you did not present a key document. When they return your documents you find the key document included in your returned application. No right to appeal, you have just had a serious refusal stamp on your passport. This is just one example of many.
So if the UK leaves the EU, and puts back visa checks to the EU citizens, that means a lot of people will have a hard time going to the UK and no one will be able to work in the UK without a Tier 2 visa, that subjects the employers to heavy scrutiny of the Home office again. Because of the scrutiny and requirements of this general work visa, even right now there is only a limited number of companies that hold a Tier 2 sponsorship.
In this regard, I honestly think, unless they amend the visa system making it lenient, they can seriously cut down the number of immigrant workers.
Now is this a bad thing or good thing? I leave it to you, but it makes the UK one of the most isolationist countries in this regard.
Many British citizens do not know how the immigration law has been changed in the last 5 years. People think home office does not do anything but they actually took such harsh steps that is beyond the human rights of immigrants, removing the right to appeal being the biggest issue. These measures do not effect the EU immigrants, which is the largest numbers of immigrants naturally. If the EU immigrants become subject to the third country national regime then they can seriously cut the number of immigration down.
Though, Most Third country nationals are now well aware that the UK has even surpassed the USA and became one of the hardest countries to get a visa (in all aspects, for tourists visits or work...) Personally I find it strange that the UK is following the example of the DPRK when it comes to visas and immigration.
I just hope that the UK will turn to the mirror and see what it is about to become.