Paste this formula into the column to the right of your p-values, and watch it
go to work. Using this formula as is requires that Excel be set to the R1C1
reference style (see Tools_Options_General).

=IF(C[-1]>=0.1,"",IF(C[-1]>=0.05,"+",IF(C[-1]>=0.01,"*",IF(C[-1]>=0.001,"**",IF(C[-1]>=0,"***","")))))

Note that this formula produces stars for +=p<.1, *=p<.05, **=p<.01, ***=p<.001;
but it can be easily modified.

After you paste the formula into the column to the right of your p-values, you
can set the width of the p-value column to 0 to hide it.

Note that the default for numeric cells in Excel is to show leading zeros (e.g,
0.123). With the p-value conversion program, you solve the problem of typing in
stars, but you create the problem of annoying leading zeros. It is possible to
fix this in Excel, however. Go to Format -> Cells -> Category=Custom and type
.##0 in the "Field" box. That will give you three decimal places but no leading
zeros. For more or fewer decimal places, change the number of #'s.