Wa alaikum salaam wa rahmatulahi wa barakatuh,

His behaviour certainly does sound suspicious and points towards what you suggest.  As a sister, it is difficult to have a heart to heart with him about such issues and would be uncomfortable for you both most likely. Even if it is not that he is engaged in haram activities, the fact that he is locked away in his room and staying up until late is of concern regardless.  This is not a healthy lifestyle to be living.

Whilst it would be uncomfortable to talk to him about your concerns so directly, there are other ways you could address the issue that are less direct.

As his sister, it would be very appropriate to talk to him more generally about how he is doing. Try to find out if there is something that is bothering him that you could help him with. It may be that the reason he hides in his room until late is less suspicious than it seems and is a result of social difficulties and therefore, such discussions with him open the door for him to confide in you and for you to show your support.

Furthermore, engaging in such haram activities can be a result of social issues and poor friendships. Therefore, providing an opportunity for him to talk about problems he may be having socially would also be an indirect way to tackle the root cause of the potential issue. It may be that he shares very little to begin with, if anything, but in time as he comes to feel more comfortable with you realizing your genuine concern for him, he will open up more and more.

Otherwise, you might approach someone who you know that he responds well to and ask them to have the same conversation and show some support for him that he may open up to them and confide in them about what is going on.

Alternatively, without even talking to him about any problems he might be having, supporting him in engaging in more useful activities will not only be good for his wellbeing but it will keep him engaged in useful things that helps to prevent involving oneself in haram activities.

The most beneficial of such activities are those that are engrained in Islamic values, such as involving oneself in tasks at the masjid. This promotes the development of friendships with other brothers who will be a positive influence to him. It will also help to promote a sense of Allah consciousness. This way, without even approaching the issue of what he could be doing directly when he has developed a stronger sense of Allah consciousness, the fear of Allah and consequences for doing haram things will be enough to push him away from doing such things himself without being told to and having to face such embarrassment.

May Allah reward your concern for your brother and guide him on the straight path.