By the way and incidentally are used to introduce something that is not directly connected with the conversation.
- I met John yesterday and we talked about his new projects. Oh, by the way, he sends you his regards.
- Jane is very upset with Peter. She thinks that he shouldn’t have hid anything from her. Incidentally, she has lost her job.
These two expressions can also be used to change the subject completely.
- ‘James has had another stroke’. ‘Oh, yes? Poor guy. By the way have you seen John recently?’
- ‘Nice evening.’ ‘Yes, isn’t it? Oh, incidentally, what happened to that $50 I lent you?’