Troubleshoot streaming job not scaling down in Dataflow | C2C Community

Troubleshoot streaming job not scaling down in Dataflow

  • 29 September 2022
  • 0 replies
  • 16 views

Userlevel 7
Badge +35

Would you like to learn how to troubleshoot autoscaling issues in Dataflow ? In particular, are you observing that your streaming jobs are not scaling down? Would you like to know more about the autotuning features in Cloud Dataflow? Check out this video below to learn about the Autotuning features Horizontal autoscaling, Vertical autoscaling, and Dynamic Work Rebalancing. Watch this video to learn about the possible causes for streaming jobs not scaling down and how to resolve the same. It also covers the streaming engine and its benefits.

Click on video below to watch it in detail:

 

Chapters:

0:00 - Intro

0:16 - Autotuning features

1:44 - Problem: Streaming job not scaling down

1:59 - Validation of the criteria for autoscaling

2:20 - Limitations of streaming appliance

2:34 - Example of limitations imposed by streaming appliance

3:05 - Cause for streaming job not scaling down

3:20 - Solution for the problem

3:33 - Benefits of using the streaming engine

3:57 - Documentation and conclusion

 

Extra credit:


0 replies

Be the first to reply!

Reply