Preheat oven to 275°F. Arrange bread cubes on a baking sheet and bake until completely dried, about 45 minutes, then transfer to a large bowl. Increase oven temperature to 350°F.
Heat a large skillet on medium heat. Add butter and chorizo, and cook until sausage is crisp and fat is reddish/orange. Stir in paprika, rosemary, and onions, then cook until onions turn translucent. Stir in garlic and olives and cook for one minute until fragrant.
Add stock and saffron. Stir to combine, then bring mixture to a simmer. Let cook for five minutes at a bare simmer for flavors to meld. Add salt and sugar as needed. Remove rosemary.
Pour broth mixture over bread cubes in mixing bowl and toss with a spatula until well combined. Transfer mixture to a 9 by 13 baking dish, lightly distributing evenly; do not pack down. Bake for 30 minutes or until top is well browned but center is still moist.