What usually happened after our submarines attacked Japanese ships?

For some reason, that seemed to make the Japanese angry and they would respond. Sometimes the boats were fortunate and, in the chaos that followed, the Japanese would drop depth charges indiscriminately. They might not be anywhere near the attacking boat. In those cases, the Japanese seemed to be hoping mostly just discourage another attack or to scare the submarines away.

Other times, the counter attacks were very effective. Remember that we did lose 52 submarines during the war and at least 27 were due to Japanese anti-submarine warfare forces.