Deleting a group is slightly different, depending on whether you are the creator or a member.
If you are the CREATOR of a group, when you press the Delete Group button on the bottom left of the Group screen it removes you and all other members from the group. The group will no longer be visible on your phone, and the group is no longer joinable by anyone, either by invitation or code. It also removes the group message conversation from your phone. The group message thread will still be viewable on other members' phones, but they will no longer be able to send messages to the group.
If you are a MEMBER of a group and tap on the Delete Group button on the bottom left of the Group screen or the (X) in the top right corner of the group in the groups list, it removes you from the group. The group will no longer be visible on your phone and it removes the group message conversation from your phone. If the creator deletes the group, you will be removed from the group and returned to the initial group screen. The group will no longer be available to join. The group message thread will still be viewable in the Messages screen, although you will be unable to send any more messages to the group.