Select column values linked to exact set of values in another column

Given the below chats_users table in Postgres 9.5, with integer columns chat_id and user_id, I want to find the chat_id associated with an exact set of user_ids, for example:

user_id IN (1, 3)    => chat_id 2
user_id IN (1, 2, 3) => chat_id 1
user_id IN (1, 2)    => no result

Is there a simple join to return the correct chat_id?

|-----------------------|
| chats_users           |
|-----------|-----------|
| chat_id 1 | user_id 1 |
| chat_id 1 | user_id 2 |
| chat_id 1 | user_id 3 |
| chat_id 2 | user_id 1 |
| chat_id 2 | user_id 3 |
| chat_id 3 | user_id 2 |
| chat_id 3 | user_id 3 |
|-----------------------|