Yes, it is difficult. Any phase of one's life is difficult, but if you persist in trying to do something, it will become less hard.

Getting rid of diapers is difficult, but everybody does it in a course of one, two or even three years, ok... 4, it happens - but it is over, and you go on.

The same is with languages: depending on your surroundings: whether Serbian is spoken every day or you are completely on your own, whether you are a fast learner or not...

Depending on numerous factors like these, the process can take a year, two, or even three, ok... sometimes 4, but you will learn this language and you are going to love it :-)