When you accidentally rejected/deleted a comment, you can only do something about it if you are moderating comments via email notification. If you moderate via the Dashboard, the moment you reject a comment, it is gone forever. On the other hand, if you moderate comments via email notification, the comment will still be in the Inbox together with a link to the commenter's profile if he/she has made his profile public.
This is how you can put back a comment you accidentally rejected. If you have set your comments to "Only Registered Users", temporarily set it to enable anonymous comments (you can always reset it back again later). Then in the email, click on the link to the post where the comment was posted, scroll down to the comments section and click "Post a Comment". There will be 3 options:
1. Blogger account (which will display your name if you are signed in
Tick "Other" and this will be what you see:
In the email notification for comment, right-click on the profile link of the commenter and select "Copy" to copy the URL of the profile page, then paste it into the field for "Your Web Page". Type the display name of the commenter in the field for "Name". Copy the comment from the email notification for comment and paste it into the "Leave your comment" field, preview, and if satisfied, click "Publish your comment". The only thing is, the profile photo will not be displayed.
You can also use this "trick" to edit a comment by first copying the comment, editing it, delete the original comment, then repost it using the method described above.
This trick is open to abuse and I have reported it to Blogger, requesting them to remove the option for OTHER, but I doubt they will do anything. In any case, I sincerely hope you don't abuse this trick.
If you don't want people to use your blog for abuse, I think it is wise to set enable comment moderation and/or to only allow registered users to comment. The disadvantage of this of course is those without Blogger account and/or don't want to register for one will be unable to comment in your blog.