Yes, you can accept a declined invitation and start the communication.
You may also like to see :
Can I decline a member after having accepted him/her?