Another scenario is: why not to wait for the external service to wake up so that we can resume the processing. This way we can avoid one drawback of the earlier approach. which is as follow: for one entity we got error and we pushed it to dlt. but we got another message for the same entity and this was processed successfully. now when the dlt msgs will be processed, this will update the entity as per previous data which will create data inconsistency. Waiting for the service to wake up will ensure two things. 1. safe guard the chronology of the events 2. no unnecessary consumption and retry and then publishing to dlt. This is my observation. I would like to hear from you on this. Thank you sir.
Hello. Thank you for your clear explanation. When i tired in my local machine with the Retry and DLT mentioned configurations with the spring Boot 3.3.2 + kafka(In built version 3.7.1) version the excepted output is not working. Can you please help me on that.
This guy literally explains everything in very simple way 👍
Exceptional....eas never knowing about DLT ...... really made my day with feeling that i learned something new today....i anyways always keep watching many of ur posted vidoes.... thanks for ur efforts for sharing ur knowledge
What do we do with this DLT list of data, do we address them manually or any standard solutions to perform in real time prod projects?
Another real time video from you sir. Thank so much sir for your hard works
Hey guys, I need to implement a retries when producing to Kafka and its related tests. Do you have references to accomplish this?
I am not aware of this. Thank you soo much for all your videos.
Hey thanks for the video! So we have topic called for example, "myTopic" and a DLT set up like this "myTopic.DLT". It is my undertsanding that kafka will just add "-DLT" to the end of your topic name is that correct? and if so is there a way to make it add ".DLT" instead? It was a another team that named them so we have to work around that.
what happened to DLT-topic when exception record is recorded inside it ? Do programmer need to manually retry from this topic or it is taken care by Kafka?
Great work , exactly what i have been looking for , thanks a lot for the hard work in bringing this tutorial .
Thank you so much for your great work. I completed this series!
@Javatechie how to process messages in DLT ? do we need another consumer to process messages from DLT? how Kafka knows whether the message is processed or not?
Is the implementation and configuration same for Kafka producer ?
Appreciate your efforts Basant. God bless you❤😊every week waiting for new updates…
thank you sir for your clear explanation. I have one question here why we are creating multiple retry topics here although we already have DLT topic to track the failure message. Can't we reuse the same topic for retry?
If the topic was consumed by multiple consumer groups, will retry/dlt be created for each consumer group?
Thanks a lot for these amazing tutorials! I learned a lot from your videos.
Nice, but why you are using Producer and Consumer separately, if we use KafkaStream then it will automatically handle both scenarios
@dheereshsingh6415