Heat the oil in a large saucepan over medium-high heat until hot. Sprinkle the chicken with 1/4 teaspoon each salt and pepper and add to the hot oil. Cook, stirring, until the chicken is no longer pink on the outside but is not cooked through, about 2 minutes. Transfer to a bowl with a slotted spoon, reserving the oil in the pan.
Stir in the squash, celery and onions and cook, stirring occasionally, until the onions are soft, about 5 minutes. Add the broth, dill sprigs and 3 cups water. Bring to a boil, and then reduce the heat to medium to maintain a steady simmer. Simmer until a knife easily pierces through the squash, about 15 minutes.
Meanwhile, whisk together the flour, baking soda, 1/4 teaspoon salt and 1/8 teaspoon pepper in a small bowl. With your fingertips, cut the butter into the flour until the mixture forms coarse crumbs. Stir in the milk just until a dough forms.
Stir the chicken and peas into the broth and bring to a simmer. Drop the dough into the simmering broth by teaspoons, spacing them evenly to make about 16 dumplings. Cover and simmer on medium low for 5 minutes. Uncover and simmer until the dumplings are cooked through, about 3 minutes. Discard the dill sprigs. Top with chopped dill and serve.