Sort:  

Ok just found by trying that if you keep http-server-address setting - RPC will run on both Http and Https protocols using ports specified for each.
You can not uncomment http-server-address setting but you can empty it's value to disable Http access:
http-server-address =

You can leave both protocols enabled. But if you want to disable HTTP, you have to set http-server-address = (commenting out the line will enable on default port)